2023-11-24T07:45:50.697 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T07:45:50.697 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578 --owner scheduled_rishabh@teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} --verbose -- /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/orig.config.yaml 2023-11-24T07:45:50.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T07:45:50.832 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} email: null first_in_suite: false job_id: '7465578' kernel: client: sha1: distro flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-tests sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9764 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-tests suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - check-counter: counters: mgr: - min: 4 name: finisher-volumes.complete_latency.avgcount - expected_val: 0 name: finisher-volumes.queue_len - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestPerModuleFinsherThread teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_07:31:48 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T07:45:50.832 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa; will attempt to use it 2023-11-24T07:45:50.833 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks 2023-11-24T07:45:50.833 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T07:45:50.835 INFO:teuthology.task.internal:Checking packages... 2023-11-24T07:45:50.860 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T07:45:50.861 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T07:45:50.861 INFO:teuthology.packaging:ref: None 2023-11-24T07:45:50.861 INFO:teuthology.packaging:tag: None 2023-11-24T07:45:50.861 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T07:45:50.861 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:45:50.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T07:45:51.023 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T07:45:51.023 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T07:45:51.030 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T07:45:51.030 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T07:45:51.125 INFO:teuthology.task.internal:Saving configuration 2023-11-24T07:45:51.143 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T07:45:51.148 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T07:45:51.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:40:55.545724', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T07:45:51.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:40:55.544288', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T07:45:51.196 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T07:45:51.202 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T07:45:51.202 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T07:45:51.202 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T07:45:51.274 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd4910d1b80>, signals=[15]) 2023-11-24T07:45:51.274 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T07:45:51.280 INFO:teuthology.task.internal:Opening connections... 2023-11-24T07:45:51.280 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:45:51.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:45:51.349 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:45:51.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:45:51.416 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T07:45:51.424 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-11-24T07:45:51.444 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-11-24T07:45:51.445 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-11-24T07:45:51.503 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T07:45:51.503 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T07:45:51.503 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-11-24T07:45:51.503 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T07:45:51.504 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T07:45:51.505 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T07:45:51.505 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-11-24T07:45:51.530 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-11-24T07:45:51.551 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-11-24T07:45:51.551 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-11-24T07:45:51.610 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T07:45:51.611 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:45:51.612 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:45:51.612 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T07:45:51.612 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T07:45:51.612 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T07:45:51.612 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T07:45:51.612 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-11-24T07:45:51.637 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T07:45:51.647 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T07:45:51.652 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T07:45:51.653 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:45:51.655 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:45:51.674 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T07:45:51.680 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T07:45:51.680 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:45:51.715 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:45:51.743 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T07:45:51.799 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T07:45:51.814 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T07:45:51.814 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-24T07:45:51.814 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-24T07:45:51.814 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:45:51.815 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:45:51.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:45:51.834 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-11-24T07:45:51.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:45:51.835 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2023-11-24T07:45:51.893 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T07:45:51.894 INFO:teuthology.task.kernel:Running kernel on smithi032: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:45:51.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2023-11-24T07:45:51.895 INFO:teuthology.orchestra.run.smithi123.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T07:45:51.895 INFO:teuthology.task.kernel:Running kernel on smithi123: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:45:51.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2023-11-24T07:45:53.102 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-11-24T07:45:53.102 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2023-11-24T07:45:53.149 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-11-24T07:45:53.149 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-11-24T07:45:54.466 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-24T07:45:54.588 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-11-24T07:45:55.192 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-24T07:45:55.555 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-11-24T07:45:58.684 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 289 kB/s | 24 kB 00:00 2023-11-24T07:45:59.256 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 469 kB/s | 24 kB 00:00 2023-11-24T07:45:59.370 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T07:45:59.430 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-11-24T07:45:59.431 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2023-11-24T07:45:59.432 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-11-24T07:45:59.556 DEBUG:teuthology.orchestra.run.smithi123:> echo no | sudo yum reinstall kernel || true 2023-11-24T07:45:59.894 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-11-24T07:45:59.894 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2023-11-24T07:46:00.017 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T07:46:00.066 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-11-24T07:46:00.067 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2023-11-24T07:46:00.068 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-11-24T07:46:00.160 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 07:45:58 AM UTC. 2023-11-24T07:46:00.170 DEBUG:teuthology.orchestra.run.smithi032:> echo no | sudo yum reinstall kernel || true 2023-11-24T07:46:00.206 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:46:00.239 INFO:teuthology.orchestra.run.smithi123.stderr:Error: No packages marked for reinstall. 2023-11-24T07:46:00.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum reinstall -y kernel || true 2023-11-24T07:46:00.561 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-11-24T07:46:00.561 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-11-24T07:46:00.612 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-11-24T07:46:00.612 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2023-11-24T07:46:00.833 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:01 ago on Fri 24 Nov 2023 07:45:59 AM UTC. 2023-11-24T07:46:00.866 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 07:45:58 AM UTC. 2023-11-24T07:46:00.875 INFO:teuthology.orchestra.run.smithi032.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:46:00.908 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:46:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:Error: No packages marked for reinstall. 2023-11-24T07:46:00.940 INFO:teuthology.orchestra.run.smithi123.stderr:Error: No packages marked for reinstall. 2023-11-24T07:46:00.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum reinstall -y kernel || true 2023-11-24T07:46:00.986 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T07:46:01.066 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T07:46:01.066 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:46:01.067 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T07:46:01.067 DEBUG:teuthology.parallel:result is None 2023-11-24T07:46:01.315 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-11-24T07:46:01.316 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-11-24T07:46:01.584 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 07:45:59 AM UTC. 2023-11-24T07:46:01.628 INFO:teuthology.orchestra.run.smithi032.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:46:01.650 INFO:teuthology.orchestra.run.smithi032.stderr:Error: No packages marked for reinstall. 2023-11-24T07:46:01.702 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T07:46:01.785 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T07:46:01.785 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:46:01.785 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T07:46:01.785 DEBUG:teuthology.parallel:result is None 2023-11-24T07:46:01.785 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T07:46:01.792 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T07:46:01.793 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:46:01.795 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:46:01.813 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T07:46:01.819 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T07:46:01.853 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T07:46:01.854 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:46:01.856 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:46:01.891 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T07:46:01.897 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T07:46:01.898 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:46:01.927 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:46:01.965 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:46:01.966 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:46:01.977 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:46:02.005 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:46:02.007 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T07:46:02.014 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T07:46:02.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:46:02.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:46:02.076 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T07:46:02.084 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T07:46:02.085 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:46:02.088 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:46:02.133 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:46:02.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:46:02.233 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:46:02.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:46:02.360 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:46:02.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:46:02.428 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:46:02.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:46:02.527 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:46:02.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:46:02.655 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:46:02.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:46:02.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-11-24T07:46:02.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-11-24T07:46:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:46:02.843 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:46:03.221 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T07:46:03.228 INFO:teuthology.task.internal:Starting timer... 2023-11-24T07:46:03.228 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T07:46:03.238 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T07:46:03.246 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-24T07:46:03.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2023-11-24T07:46:03.310 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2023-11-24T07:46:03.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2023-11-24T07:46:03.370 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2023-11-24T07:46:03.372 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T07:46:03.372 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2023-11-24T07:46:03.394 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2023-11-24T07:46:03.395 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2023-11-24T07:46:03.414 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2023-11-24T07:46:03.415 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2023-11-24T07:46:03.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:46:03.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:46:03.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:46:03.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:46:03.490 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T07:46:03.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:46:03.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:46:03.833 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T07:46:03.843 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-24T07:46:03.849 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-24T07:46:03.982 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T07:46:03.997 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T07:46:03.998 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T07:55:21.530 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2023-11-24T07:55:21.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-11-24T07:55:21.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:55:21.601 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-11-24T07:55:21.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-11-24T07:55:21.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-11-24T07:55:21.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:55:21.741 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-11-24T07:55:21.811 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-11-24T07:55:21.811 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T07:55:21.824 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T07:55:21.825 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T07:55:21.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T07:55:21.827 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T07:55:21.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T07:55:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T07:55:21.875 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T07:55:21.881 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T07:55:21.896 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T07:55:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2023-11-24T07:55:21.961 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2023-11-24T07:55:21.975 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2023-11-24T07:55:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T07:55:21.986 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2023-11-24T07:55:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T07:55:22.001 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T07:55:22.014 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T07:55:22.057 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-11-24T07:55:22.060 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T07:55:22.060 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-11-24T07:55:22.060 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.060 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.061 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.061 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.086 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-11-24T07:55:22.089 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T07:55:22.089 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-11-24T07:55:22.089 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.090 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.090 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.090 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:55:22.091 INFO:teuthology.run_tasks:Running task install... 2023-11-24T07:55:22.099 DEBUG:teuthology.task.install:project ceph 2023-11-24T07:55:22.099 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T07:55:22.100 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T07:55:22.100 INFO:teuthology.task.install:Using flavor: default 2023-11-24T07:55:22.105 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T07:55:22.106 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T07:55:22.106 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T07:55:22.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:55:22.108 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T07:55:22.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:55:22.251 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T07:55:22.251 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T07:55:22.327 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T07:55:22.327 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T07:55:22.439 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T07:55:22.439 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:55:22.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T07:55:22.441 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T07:55:22.442 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:55:22.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T07:55:22.470 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T07:55:22.470 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T07:55:22.472 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T07:55:22.472 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T07:55:22.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T07:55:22.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T07:55:22.615 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T07:55:22.623 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T07:55:22.685 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2023-11-24T07:55:22.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2023-11-24T07:55:22.694 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2023-11-24T07:55:22.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2023-11-24T07:55:23.057 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-11-24T07:55:23.093 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-11-24T07:55:23.346 INFO:teuthology.orchestra.run.smithi123.stdout:44 files removed 2023-11-24T07:55:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:44 files removed 2023-11-24T07:55:23.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T07:55:23.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T07:55:23.722 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-11-24T07:55:23.761 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-11-24T07:55:24.418 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 331 kB/s | 81 kB 00:00 2023-11-24T07:55:24.467 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 211 kB/s | 81 kB 00:00 2023-11-24T07:55:24.727 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 34 kB/s | 9.5 kB 00:00 2023-11-24T07:55:24.759 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 36 kB/s | 9.5 kB 00:00 2023-11-24T07:55:25.019 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-11-24T07:55:25.094 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 4.8 kB/s | 1.5 kB 00:00 2023-11-24T07:55:25.262 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-11-24T07:55:25.311 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-11-24T07:55:26.045 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 6.0 MB/s | 16 MB 00:02 2023-11-24T07:55:29.637 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 355 kB/s | 24 kB 00:00 2023-11-24T07:55:30.597 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-11-24T07:55:31.679 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 387 kB/s | 24 kB 00:00 2023-11-24T07:55:32.722 INFO:teuthology.orchestra.run.smithi032.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 MB/s | 53 MB 00:00 2023-11-24T07:55:38.906 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-11-24T07:55:41.365 INFO:teuthology.orchestra.run.smithi032.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-11-24T07:55:49.815 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:55:49.816 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:55:49.831 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T07:55:49.832 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T07:55:49.833 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T07:55:49.834 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T07:55:49.836 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T07:55:49.839 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T07:55:49.840 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T07:55:49.971 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-11-24T07:55:49.983 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-11-24T07:55:49.983 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-11-24T07:55:49.983 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-11-24T07:55:49.983 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T07:55:49.984 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T07:55:49.985 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T07:55:49.986 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T07:55:49.987 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T07:55:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T07:55:49.989 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T07:55:49.990 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T07:55:49.993 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-11-24T07:55:49.994 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql 10 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout:Install 128 Packages 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 3 Packages 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:55:49.995 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 209 M 2023-11-24T07:55:49.996 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-11-24T07:55:50.331 INFO:teuthology.orchestra.run.smithi123.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-11-24T07:55:50.722 INFO:teuthology.orchestra.run.smithi123.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 2.3 MB/s | 920 kB 00:00 2023-11-24T07:55:50.823 INFO:teuthology.orchestra.run.smithi123.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 1.7 MB/s | 171 kB 00:00 2023-11-24T07:55:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:(4/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 4.0 MB/s | 2.3 MB 00:00 2023-11-24T07:55:51.523 INFO:teuthology.orchestra.run.smithi123.stdout:(5/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 3.9 MB/s | 5.4 MB 00:01 2023-11-24T07:55:51.790 INFO:teuthology.orchestra.run.smithi123.stdout:(6/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 4.2 MB/s | 1.6 MB 00:00 2023-11-24T07:55:51.990 INFO:teuthology.orchestra.run.smithi123.stdout:(7/131): ceph-common-18.0.0-7484.ga13ea5ff.el8. 12 MB/s | 23 MB 00:01 2023-11-24T07:55:52.199 INFO:teuthology.orchestra.run.smithi123.stdout:(8/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 6.9 MB/s | 4.6 MB 00:00 2023-11-24T07:55:52.325 INFO:teuthology.orchestra.run.smithi123.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 194 kB/s | 24 kB 00:00 2023-11-24T07:55:52.916 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:55:52.917 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:55:52.932 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T07:55:52.933 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T07:55:52.934 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T07:55:52.935 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T07:55:52.937 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T07:55:52.941 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T07:55:52.942 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T07:55:53.089 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-11-24T07:55:53.102 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-11-24T07:55:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-11-24T07:55:53.102 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-11-24T07:55:53.102 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2023-11-24T07:55:53.102 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T07:55:53.103 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T07:55:53.104 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T07:55:53.105 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T07:55:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T07:55:53.107 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T07:55:53.108 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T07:55:53.109 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T07:55:53.110 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T07:55:53.111 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:55:53.112 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T07:55:53.113 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling module streams: 2023-11-24T07:55:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql 10 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Install 128 Packages 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 3 Packages 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 209 M 2023-11-24T07:55:53.116 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-11-24T07:55:53.464 INFO:teuthology.orchestra.run.smithi123.stdout:(10/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 9.4 MB/s | 14 MB 00:01 2023-11-24T07:55:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 1.6 MB/s | 262 kB 00:00 2023-11-24T07:55:53.684 INFO:teuthology.orchestra.run.smithi123.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 534 kB/s | 31 kB 00:00 2023-11-24T07:55:53.690 INFO:teuthology.orchestra.run.smithi032.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2023-11-24T07:55:53.926 INFO:teuthology.orchestra.run.smithi123.stdout:(13/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x8 8.4 MB/s | 18 MB 00:02 2023-11-24T07:55:54.009 INFO:teuthology.orchestra.run.smithi123.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.2 MB/s | 183 kB 00:00 2023-11-24T07:55:54.093 INFO:teuthology.orchestra.run.smithi123.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 1.6 MB/s | 139 kB 00:00 2023-11-24T07:55:54.118 INFO:teuthology.orchestra.run.smithi123.stdout:(16/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 1.9 MB/s | 839 kB 00:00 2023-11-24T07:55:54.149 INFO:teuthology.orchestra.run.smithi032.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 2.0 MB/s | 920 kB 00:00 2023-11-24T07:55:54.185 INFO:teuthology.orchestra.run.smithi123.stdout:(17/131): libradosstriper1-18.0.0-7484.ga13ea5f 5.5 MB/s | 517 kB 00:00 2023-11-24T07:55:54.260 INFO:teuthology.orchestra.run.smithi123.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 596 kB/s | 45 kB 00:00 2023-11-24T07:55:54.299 INFO:teuthology.orchestra.run.smithi032.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 1.1 MB/s | 171 kB 00:00 2023-11-24T07:55:54.344 INFO:teuthology.orchestra.run.smithi123.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 1.5 MB/s | 128 kB 00:00 2023-11-24T07:55:54.436 INFO:teuthology.orchestra.run.smithi123.stdout:(20/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 2.1 MB/s | 197 kB 00:00 2023-11-24T07:55:54.544 INFO:teuthology.orchestra.run.smithi123.stdout:(21/131): python3-rados-18.0.0-7484.ga13ea5ff.e 3.3 MB/s | 371 kB 00:00 2023-11-24T07:55:54.653 INFO:teuthology.orchestra.run.smithi123.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 3.3 MB/s | 364 kB 00:00 2023-11-24T07:55:54.753 INFO:teuthology.orchestra.run.smithi123.stdout:(23/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 1.0 MB/s | 107 kB 00:00 2023-11-24T07:55:54.862 INFO:teuthology.orchestra.run.smithi123.stdout:(24/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 863 kB/s | 94 kB 00:00 2023-11-24T07:55:55.266 INFO:teuthology.orchestra.run.smithi032.stdout:(4/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-11-24T07:55:55.400 INFO:teuthology.orchestra.run.smithi032.stdout:(5/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 2.8 MB/s | 5.4 MB 00:01 2023-11-24T07:55:55.521 INFO:teuthology.orchestra.run.smithi123.stdout:(25/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 5.4 MB/s | 3.6 MB 00:00 2023-11-24T07:55:55.629 INFO:teuthology.orchestra.run.smithi123.stdout:(26/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 1.6 MB/s | 181 kB 00:00 2023-11-24T07:55:55.721 INFO:teuthology.orchestra.run.smithi123.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 304 kB/s | 28 kB 00:00 2023-11-24T07:55:55.822 INFO:teuthology.orchestra.run.smithi123.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.3 MB/s | 133 kB 00:00 2023-11-24T07:55:55.967 INFO:teuthology.orchestra.run.smithi032.stdout:(6/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 2.3 MB/s | 1.6 MB 00:00 2023-11-24T07:55:56.205 INFO:teuthology.orchestra.run.smithi123.stdout:(29/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 4.6 MB/s | 1.8 MB 00:00 2023-11-24T07:55:56.242 INFO:teuthology.orchestra.run.smithi032.stdout:(7/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 5.5 MB/s | 4.6 MB 00:00 2023-11-24T07:55:56.855 INFO:teuthology.orchestra.run.smithi123.stdout:(30/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 2.5 MB/s | 6.9 MB 00:02 2023-11-24T07:55:57.031 INFO:teuthology.orchestra.run.smithi123.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 1.4 MB/s | 243 kB 00:00 2023-11-24T07:55:57.131 INFO:teuthology.orchestra.run.smithi123.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 501 kB/s | 50 kB 00:00 2023-11-24T07:55:57.231 INFO:teuthology.orchestra.run.smithi123.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 150 kB/s | 15 kB 00:00 2023-11-24T07:55:57.468 INFO:teuthology.orchestra.run.smithi123.stdout:(34/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 1.1 MB/s | 262 kB 00:00 2023-11-24T07:55:57.691 INFO:teuthology.orchestra.run.smithi123.stdout:(35/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 1.8 MB/s | 407 kB 00:00 2023-11-24T07:55:57.759 INFO:teuthology.orchestra.run.smithi032.stdout:(8/131): ceph-common-18.0.0-7484.ga13ea5ff.el8. 5.2 MB/s | 23 MB 00:04 2023-11-24T07:55:57.909 INFO:teuthology.orchestra.run.smithi032.stdout:(9/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el8 8.3 MB/s | 14 MB 00:01 2023-11-24T07:55:57.935 INFO:teuthology.orchestra.run.smithi032.stdout:(10/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el 139 kB/s | 24 kB 00:00 2023-11-24T07:55:58.035 INFO:teuthology.orchestra.run.smithi032.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.6 MB/s | 262 kB 00:00 2023-11-24T07:55:58.093 INFO:teuthology.orchestra.run.smithi032.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 533 kB/s | 31 kB 00:00 2023-11-24T07:55:58.141 INFO:teuthology.orchestra.run.smithi123.stdout:(36/131): ceph-test-18.0.0-7484.ga13ea5ff.el8.x 8.5 MB/s | 49 MB 00:05 2023-11-24T07:55:58.166 INFO:teuthology.orchestra.run.smithi123.stdout:(37/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 38 kB/s | 18 kB 00:00 2023-11-24T07:55:58.250 INFO:teuthology.orchestra.run.smithi123.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 3.6 MB/s | 7.4 MB 00:02 2023-11-24T07:55:58.277 INFO:teuthology.orchestra.run.smithi032.stdout:(13/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 4.5 MB/s | 839 kB 00:00 2023-11-24T07:55:58.325 INFO:teuthology.orchestra.run.smithi123.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 371 kB/s | 59 kB 00:00 2023-11-24T07:55:58.350 INFO:teuthology.orchestra.run.smithi123.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2023-11-24T07:55:58.361 INFO:teuthology.orchestra.run.smithi032.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.2 MB/s | 183 kB 00:00 2023-11-24T07:55:58.384 INFO:teuthology.orchestra.run.smithi123.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-24T07:55:58.409 INFO:teuthology.orchestra.run.smithi123.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 745 kB/s | 44 kB 00:00 2023-11-24T07:55:58.428 INFO:teuthology.orchestra.run.smithi032.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 2.0 MB/s | 139 kB 00:00 2023-11-24T07:55:58.434 INFO:teuthology.orchestra.run.smithi123.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-11-24T07:55:58.460 INFO:teuthology.orchestra.run.smithi123.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 658 kB/s | 33 kB 00:00 2023-11-24T07:55:58.485 INFO:teuthology.orchestra.run.smithi123.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-11-24T07:55:58.519 INFO:teuthology.orchestra.run.smithi123.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2023-11-24T07:55:58.544 INFO:teuthology.orchestra.run.smithi123.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-11-24T07:55:58.553 INFO:teuthology.orchestra.run.smithi032.stdout:(16/131): libradosstriper1-18.0.0-7484.ga13ea5f 4.0 MB/s | 517 kB 00:00 2023-11-24T07:55:58.569 INFO:teuthology.orchestra.run.smithi123.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-11-24T07:55:58.586 INFO:teuthology.orchestra.run.smithi123.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2023-11-24T07:55:58.603 INFO:teuthology.orchestra.run.smithi123.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 525 kB/s | 18 kB 00:00 2023-11-24T07:55:58.653 INFO:teuthology.orchestra.run.smithi123.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-11-24T07:55:58.670 INFO:teuthology.orchestra.run.smithi123.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 730 kB/s | 49 kB 00:00 2023-11-24T07:55:58.687 INFO:teuthology.orchestra.run.smithi123.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-11-24T07:55:58.704 INFO:teuthology.orchestra.run.smithi123.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-11-24T07:55:58.729 INFO:teuthology.orchestra.run.smithi123.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2023-11-24T07:55:58.746 INFO:teuthology.orchestra.run.smithi123.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 374 kB/s | 16 kB 00:00 2023-11-24T07:55:58.765 INFO:teuthology.orchestra.run.smithi123.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-11-24T07:55:58.797 INFO:teuthology.orchestra.run.smithi123.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 5.5 MB/s | 1.2 MB 00:00 2023-11-24T07:55:58.814 INFO:teuthology.orchestra.run.smithi123.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-11-24T07:55:58.831 INFO:teuthology.orchestra.run.smithi123.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 560 kB/s | 19 kB 00:00 2023-11-24T07:55:58.856 INFO:teuthology.orchestra.run.smithi123.stdout:(61/131): python3-rsa-4.9-2.el8.noarch.rpm 694 kB/s | 64 kB 00:00 2023-11-24T07:55:58.883 INFO:teuthology.orchestra.run.smithi123.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-11-24T07:55:58.898 INFO:teuthology.orchestra.run.smithi123.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 436 kB/s | 29 kB 00:00 2023-11-24T07:55:58.915 INFO:teuthology.orchestra.run.smithi123.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 825 kB/s | 27 kB 00:00 2023-11-24T07:55:58.941 INFO:teuthology.orchestra.run.smithi123.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 4.2 MB/s | 255 kB 00:00 2023-11-24T07:55:58.966 INFO:teuthology.orchestra.run.smithi123.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-24T07:55:58.991 INFO:teuthology.orchestra.run.smithi123.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-24T07:55:59.017 INFO:teuthology.orchestra.run.smithi123.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2023-11-24T07:55:59.042 INFO:teuthology.orchestra.run.smithi123.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-11-24T07:55:59.067 INFO:teuthology.orchestra.run.smithi123.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-24T07:55:59.117 INFO:teuthology.orchestra.run.smithi123.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-11-24T07:55:59.151 INFO:teuthology.orchestra.run.smithi123.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 431 kB/s | 47 kB 00:00 2023-11-24T07:55:59.185 INFO:teuthology.orchestra.run.smithi123.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2023-11-24T07:55:59.239 INFO:teuthology.orchestra.run.smithi123.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 365 kB/s | 43 kB 00:00 2023-11-24T07:55:59.269 INFO:teuthology.orchestra.run.smithi123.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 575 kB/s | 48 kB 00:00 2023-11-24T07:55:59.294 INFO:teuthology.orchestra.run.smithi123.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.6 MB/s | 87 kB 00:00 2023-11-24T07:55:59.319 INFO:teuthology.orchestra.run.smithi123.stdout:(77/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-11-24T07:55:59.344 INFO:teuthology.orchestra.run.smithi123.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 566 kB/s | 43 kB 00:00 2023-11-24T07:55:59.369 INFO:teuthology.orchestra.run.smithi123.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-11-24T07:55:59.395 INFO:teuthology.orchestra.run.smithi123.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-11-24T07:55:59.420 INFO:teuthology.orchestra.run.smithi123.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 371 kB/s | 19 kB 00:00 2023-11-24T07:55:59.445 INFO:teuthology.orchestra.run.smithi123.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-24T07:55:59.479 INFO:teuthology.orchestra.run.smithi123.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-11-24T07:55:59.504 INFO:teuthology.orchestra.run.smithi123.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-11-24T07:55:59.538 INFO:teuthology.orchestra.run.smithi123.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.7 MB/s | 559 kB 00:00 2023-11-24T07:55:59.571 INFO:teuthology.orchestra.run.smithi123.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-24T07:55:59.597 INFO:teuthology.orchestra.run.smithi123.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-11-24T07:55:59.622 INFO:teuthology.orchestra.run.smithi123.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-11-24T07:55:59.647 INFO:teuthology.orchestra.run.smithi123.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-24T07:55:59.670 INFO:teuthology.orchestra.run.smithi032.stdout:(17/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 6.2 MB/s | 6.9 MB 00:01 2023-11-24T07:55:59.672 INFO:teuthology.orchestra.run.smithi123.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-11-24T07:55:59.689 INFO:teuthology.orchestra.run.smithi123.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-24T07:55:59.715 INFO:teuthology.orchestra.run.smithi123.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-11-24T07:55:59.740 INFO:teuthology.orchestra.run.smithi123.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-11-24T07:55:59.758 INFO:teuthology.orchestra.run.smithi032.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 507 kB/s | 45 kB 00:00 2023-11-24T07:55:59.800 INFO:teuthology.orchestra.run.smithi123.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 7.4 MB/s | 644 kB 00:00 2023-11-24T07:55:59.812 INFO:teuthology.orchestra.run.smithi032.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 2.3 MB/s | 128 kB 00:00 2023-11-24T07:55:59.832 INFO:teuthology.orchestra.run.smithi123.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.8 MB/s | 171 kB 00:00 2023-11-24T07:55:59.858 INFO:teuthology.orchestra.run.smithi123.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.4 MB/s | 138 kB 00:00 2023-11-24T07:55:59.871 INFO:teuthology.orchestra.run.smithi032.stdout:(20/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 3.3 MB/s | 197 kB 00:00 2023-11-24T07:55:59.916 INFO:teuthology.orchestra.run.smithi123.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2023-11-24T07:55:59.942 INFO:teuthology.orchestra.run.smithi123.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-11-24T07:55:59.958 INFO:teuthology.orchestra.run.smithi123.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-24T07:55:59.962 INFO:teuthology.orchestra.run.smithi032.stdout:(21/131): python3-rados-18.0.0-7484.ga13ea5ff.e 4.0 MB/s | 371 kB 00:00 2023-11-24T07:55:59.984 INFO:teuthology.orchestra.run.smithi123.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-24T07:56:00.017 INFO:teuthology.orchestra.run.smithi123.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-11-24T07:56:00.034 INFO:teuthology.orchestra.run.smithi123.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 772 kB/s | 39 kB 00:00 2023-11-24T07:56:00.063 INFO:teuthology.orchestra.run.smithi032.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 3.6 MB/s | 364 kB 00:00 2023-11-24T07:56:00.068 INFO:teuthology.orchestra.run.smithi123.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-11-24T07:56:00.101 INFO:teuthology.orchestra.run.smithi123.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-11-24T07:56:00.121 INFO:teuthology.orchestra.run.smithi032.stdout:(23/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 1.8 MB/s | 107 kB 00:00 2023-11-24T07:56:00.126 INFO:teuthology.orchestra.run.smithi123.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-11-24T07:56:00.143 INFO:teuthology.orchestra.run.smithi123.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-24T07:56:00.185 INFO:teuthology.orchestra.run.smithi123.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-24T07:56:00.188 INFO:teuthology.orchestra.run.smithi032.stdout:(24/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.4 MB/s | 94 kB 00:00 2023-11-24T07:56:00.219 INFO:teuthology.orchestra.run.smithi123.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 795 kB/s | 27 kB 00:00 2023-11-24T07:56:00.261 INFO:teuthology.orchestra.run.smithi123.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T07:56:00.295 INFO:teuthology.orchestra.run.smithi123.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-11-24T07:56:00.395 INFO:teuthology.orchestra.run.smithi123.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-11-24T07:56:00.445 INFO:teuthology.orchestra.run.smithi123.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-11-24T07:56:00.487 INFO:teuthology.orchestra.run.smithi123.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-24T07:56:00.546 INFO:teuthology.orchestra.run.smithi123.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-11-24T07:56:00.571 INFO:teuthology.orchestra.run.smithi123.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-11-24T07:56:00.596 INFO:teuthology.orchestra.run.smithi123.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-24T07:56:00.622 INFO:teuthology.orchestra.run.smithi123.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-11-24T07:56:00.647 INFO:teuthology.orchestra.run.smithi123.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-24T07:56:00.680 INFO:teuthology.orchestra.run.smithi123.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-24T07:56:00.806 INFO:teuthology.orchestra.run.smithi032.stdout:(25/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x8 3.7 MB/s | 18 MB 00:04 2023-11-24T07:56:00.822 INFO:teuthology.orchestra.run.smithi123.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 14 MB/s | 14 MB 00:00 2023-11-24T07:56:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-11-24T07:56:00.864 INFO:teuthology.orchestra.run.smithi032.stdout:(26/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 5.3 MB/s | 3.6 MB 00:00 2023-11-24T07:56:00.889 INFO:teuthology.orchestra.run.smithi123.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-24T07:56:00.956 INFO:teuthology.orchestra.run.smithi123.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2023-11-24T07:56:01.032 INFO:teuthology.orchestra.run.smithi123.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-11-24T07:56:01.099 INFO:teuthology.orchestra.run.smithi123.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-24T07:56:01.141 INFO:teuthology.orchestra.run.smithi123.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-11-24T07:56:01.199 INFO:teuthology.orchestra.run.smithi123.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-11-24T07:56:01.275 INFO:teuthology.orchestra.run.smithi123.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-11-24T07:56:01.323 INFO:teuthology.orchestra.run.smithi032.stdout:(27/131): ceph-test-18.0.0-7484.ga13ea5ff.el8.x 14 MB/s | 49 MB 00:03 2023-11-24T07:56:01.333 INFO:teuthology.orchestra.run.smithi123.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-24T07:56:01.348 INFO:teuthology.orchestra.run.smithi032.stdout:(28/131): ceph-grafana-dashboards-18.0.0-7484.g 58 kB/s | 28 kB 00:00 2023-11-24T07:56:01.373 INFO:teuthology.orchestra.run.smithi032.stdout:(29/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 319 kB/s | 181 kB 00:00 2023-11-24T07:56:01.399 INFO:teuthology.orchestra.run.smithi032.stdout:(30/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.7 MB/s | 133 kB 00:00 2023-11-24T07:56:01.490 INFO:teuthology.orchestra.run.smithi032.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 2.6 MB/s | 243 kB 00:00 2023-11-24T07:56:01.549 INFO:teuthology.orchestra.run.smithi032.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 862 kB/s | 50 kB 00:00 2023-11-24T07:56:01.616 INFO:teuthology.orchestra.run.smithi032.stdout:(33/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 6.6 MB/s | 1.8 MB 00:00 2023-11-24T07:56:01.633 INFO:teuthology.orchestra.run.smithi032.stdout:(34/131): ceph-prometheus-alerts-18.0.0-7484.ga 179 kB/s | 15 kB 00:00 2023-11-24T07:56:01.683 INFO:teuthology.orchestra.run.smithi032.stdout:(35/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 3.8 MB/s | 262 kB 00:00 2023-11-24T07:56:01.734 INFO:teuthology.orchestra.run.smithi032.stdout:(36/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 4.0 MB/s | 407 kB 00:00 2023-11-24T07:56:01.751 INFO:teuthology.orchestra.run.smithi032.stdout:(37/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 270 kB/s | 18 kB 00:00 2023-11-24T07:56:01.834 INFO:teuthology.orchestra.run.smithi123.stdout:(130/131): librados2-18.0.0-7484.ga13ea5ff.el8. 5.4 MB/s | 3.8 MB 00:00 2023-11-24T07:56:01.884 INFO:teuthology.orchestra.run.smithi123.stdout:(131/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 5.4 MB/s | 3.7 MB 00:00 2023-11-24T07:56:01.886 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:56:01.886 INFO:teuthology.orchestra.run.smithi123.stdout:Total 18 MB/s | 209 MB 00:11 2023-11-24T07:56:02.284 INFO:teuthology.orchestra.run.smithi032.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 110 kB/s | 59 kB 00:00 2023-11-24T07:56:02.501 INFO:teuthology.orchestra.run.smithi032.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 347 kB/s | 75 kB 00:00 2023-11-24T07:56:02.585 INFO:teuthology.orchestra.run.smithi032.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 360 kB/s | 306 kB 00:00 2023-11-24T07:56:02.710 INFO:teuthology.orchestra.run.smithi032.stdout:(41/131): ceph-mgr-diskprediction-local-18.0.0- 5.5 MB/s | 7.4 MB 00:01 2023-11-24T07:56:02.735 INFO:teuthology.orchestra.run.smithi032.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 290 kB/s | 44 kB 00:00 2023-11-24T07:56:02.750 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-11-24T07:56:02.835 INFO:teuthology.orchestra.run.smithi032.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-11-24T07:56:02.869 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-11-24T07:56:02.870 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-11-24T07:56:02.936 INFO:teuthology.orchestra.run.smithi032.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-11-24T07:56:03.128 INFO:teuthology.orchestra.run.smithi032.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 760 kB/s | 476 kB 00:00 2023-11-24T07:56:03.228 INFO:teuthology.orchestra.run.smithi032.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 591 kB/s | 173 kB 00:00 2023-11-24T07:56:03.337 INFO:teuthology.orchestra.run.smithi032.stdout:(47/131): python3-beautifulsoup4-4.6.3-2.el8.1. 296 kB/s | 185 kB 00:00 2023-11-24T07:56:03.404 INFO:teuthology.orchestra.run.smithi032.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 463 kB/s | 81 kB 00:00 2023-11-24T07:56:03.429 INFO:teuthology.orchestra.run.smithi032.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-11-24T07:56:03.496 INFO:teuthology.orchestra.run.smithi032.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-11-24T07:56:03.596 INFO:teuthology.orchestra.run.smithi032.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2023-11-24T07:56:03.630 INFO:teuthology.orchestra.run.smithi032.stdout:(52/131): python3-cherrypy-18.4.0-1.el8.noarch. 766 kB/s | 384 kB 00:00 2023-11-24T07:56:03.697 INFO:teuthology.orchestra.run.smithi032.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 586 kB/s | 59 kB 00:00 2023-11-24T07:56:03.730 INFO:teuthology.orchestra.run.smithi032.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 602 kB/s | 60 kB 00:00 2023-11-24T07:56:03.839 INFO:teuthology.orchestra.run.smithi032.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-11-24T07:56:03.939 INFO:teuthology.orchestra.run.smithi032.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-11-24T07:56:04.003 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-11-24T07:56:04.007 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-11-24T07:56:04.048 INFO:teuthology.orchestra.run.smithi032.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 806 kB/s | 283 kB 00:00 2023-11-24T07:56:04.157 INFO:teuthology.orchestra.run.smithi032.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 593 kB/s | 64 kB 00:00 2023-11-24T07:56:04.198 INFO:teuthology.orchestra.run.smithi032.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 759 kB/s | 196 kB 00:00 2023-11-24T07:56:04.257 INFO:teuthology.orchestra.run.smithi032.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-11-24T07:56:04.299 INFO:teuthology.orchestra.run.smithi032.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-11-24T07:56:04.383 INFO:teuthology.orchestra.run.smithi032.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-11-24T07:56:04.408 INFO:teuthology.orchestra.run.smithi032.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-11-24T07:56:04.733 INFO:teuthology.orchestra.run.smithi032.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 729 kB/s | 255 kB 00:00 2023-11-24T07:56:04.758 INFO:teuthology.orchestra.run.smithi032.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 717 kB/s | 251 kB 00:00 2023-11-24T07:56:04.842 INFO:teuthology.orchestra.run.smithi032.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-11-24T07:56:04.942 INFO:teuthology.orchestra.run.smithi032.stdout:(67/131): python3-kubernetes-11.0.0-6.el8.noarc 789 kB/s | 1.2 MB 00:01 2023-11-24T07:56:04.968 INFO:teuthology.orchestra.run.smithi032.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 408 kB/s | 85 kB 00:00 2023-11-24T07:56:04.993 INFO:teuthology.orchestra.run.smithi032.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 152 kB/s | 23 kB 00:00 2023-11-24T07:56:05.010 INFO:teuthology.orchestra.run.smithi032.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-11-24T07:56:05.061 INFO:teuthology.orchestra.run.smithi032.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-11-24T07:56:05.086 INFO:teuthology.orchestra.run.smithi032.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-11-24T07:56:05.110 INFO:teuthology.orchestra.run.smithi032.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-11-24T07:56:05.136 INFO:teuthology.orchestra.run.smithi032.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-24T07:56:05.161 INFO:teuthology.orchestra.run.smithi032.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 958 kB/s | 48 kB 00:00 2023-11-24T07:56:05.186 INFO:teuthology.orchestra.run.smithi032.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-24T07:56:05.214 INFO:teuthology.orchestra.run.smithi032.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 815 kB/s | 43 kB 00:00 2023-11-24T07:56:05.239 INFO:teuthology.orchestra.run.smithi032.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.0 MB/s | 109 kB 00:00 2023-11-24T07:56:05.265 INFO:teuthology.orchestra.run.smithi032.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-11-24T07:56:05.298 INFO:teuthology.orchestra.run.smithi032.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-11-24T07:56:05.315 INFO:teuthology.orchestra.run.smithi032.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-11-24T07:56:05.340 INFO:teuthology.orchestra.run.smithi032.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-11-24T07:56:05.382 INFO:teuthology.orchestra.run.smithi032.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2023-11-24T07:56:05.408 INFO:teuthology.orchestra.run.smithi032.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 267 kB/s | 18 kB 00:00 2023-11-24T07:56:05.441 INFO:teuthology.orchestra.run.smithi032.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-24T07:56:05.467 INFO:teuthology.orchestra.run.smithi032.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-24T07:56:05.533 INFO:teuthology.orchestra.run.smithi032.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-11-24T07:56:05.567 INFO:teuthology.orchestra.run.smithi032.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-11-24T07:56:05.618 INFO:teuthology.orchestra.run.smithi032.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-11-24T07:56:05.643 INFO:teuthology.orchestra.run.smithi032.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 794 kB/s | 60 kB 00:00 2023-11-24T07:56:05.660 INFO:teuthology.orchestra.run.smithi032.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-24T07:56:05.685 INFO:teuthology.orchestra.run.smithi032.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-11-24T07:56:05.711 INFO:teuthology.orchestra.run.smithi032.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-11-24T07:56:05.744 INFO:teuthology.orchestra.run.smithi032.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-24T07:56:05.769 INFO:teuthology.orchestra.run.smithi032.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-11-24T07:56:05.794 INFO:teuthology.orchestra.run.smithi032.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-24T07:56:05.837 INFO:teuthology.orchestra.run.smithi032.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-24T07:56:05.870 INFO:teuthology.orchestra.run.smithi032.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-24T07:56:05.912 INFO:teuthology.orchestra.run.smithi032.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-24T07:56:05.954 INFO:teuthology.orchestra.run.smithi032.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-24T07:56:05.979 INFO:teuthology.orchestra.run.smithi032.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.5 MB/s | 39 kB 00:00 2023-11-24T07:56:06.163 INFO:teuthology.orchestra.run.smithi032.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-11-24T07:56:06.213 INFO:teuthology.orchestra.run.smithi032.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-11-24T07:56:06.256 INFO:teuthology.orchestra.run.smithi032.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-24T07:56:06.298 INFO:teuthology.orchestra.run.smithi032.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-24T07:56:06.348 INFO:teuthology.orchestra.run.smithi032.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-11-24T07:56:06.516 INFO:teuthology.orchestra.run.smithi032.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 18 MB/s | 14 MB 00:00 2023-11-24T07:56:06.549 INFO:teuthology.orchestra.run.smithi032.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 816 kB/s | 27 kB 00:00 2023-11-24T07:56:06.591 INFO:teuthology.orchestra.run.smithi032.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T07:56:06.624 INFO:teuthology.orchestra.run.smithi032.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-11-24T07:56:06.725 INFO:teuthology.orchestra.run.smithi032.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-11-24T07:56:06.766 INFO:teuthology.orchestra.run.smithi032.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-11-24T07:56:06.808 INFO:teuthology.orchestra.run.smithi032.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-24T07:56:06.875 INFO:teuthology.orchestra.run.smithi032.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-11-24T07:56:06.901 INFO:teuthology.orchestra.run.smithi032.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-11-24T07:56:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.4 MB/s | 126 kB 00:00 2023-11-24T07:56:06.951 INFO:teuthology.orchestra.run.smithi032.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-11-24T07:56:06.977 INFO:teuthology.orchestra.run.smithi032.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-24T07:56:07.010 INFO:teuthology.orchestra.run.smithi032.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-24T07:56:07.052 INFO:teuthology.orchestra.run.smithi032.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-24T07:56:07.169 INFO:teuthology.orchestra.run.smithi032.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-11-24T07:56:07.211 INFO:teuthology.orchestra.run.smithi032.stdout:(122/131): thrift-0.13.0-2.el8.x86_64.rpm 783 kB/s | 1.7 MB 00:02 2023-11-24T07:56:07.253 INFO:teuthology.orchestra.run.smithi032.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 2.4 MB/s | 202 kB 00:00 2023-11-24T07:56:07.303 INFO:teuthology.orchestra.run.smithi032.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-11-24T07:56:07.370 INFO:teuthology.orchestra.run.smithi032.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-11-24T07:56:07.412 INFO:teuthology.orchestra.run.smithi032.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-11-24T07:56:07.463 INFO:teuthology.orchestra.run.smithi032.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-11-24T07:56:07.546 INFO:teuthology.orchestra.run.smithi032.stdout:(128/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-11-24T07:56:07.605 INFO:teuthology.orchestra.run.smithi032.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-24T07:56:07.930 INFO:teuthology.orchestra.run.smithi032.stdout:(130/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 7.9 MB/s | 3.7 MB 00:00 2023-11-24T07:56:08.205 INFO:teuthology.orchestra.run.smithi032.stdout:(131/131): librados2-18.0.0-7484.ga13ea5ff.el8. 4.8 MB/s | 3.8 MB 00:00 2023-11-24T07:56:08.209 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:56:08.210 INFO:teuthology.orchestra.run.smithi032.stdout:Total 14 MB/s | 209 MB 00:15 2023-11-24T07:56:09.071 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-11-24T07:56:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-11-24T07:56:09.204 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-11-24T07:56:09.204 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-11-24T07:56:09.359 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T07:56:09.542 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T07:56:09.745 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T07:56:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T07:56:10.078 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T07:56:10.405 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-11-24T07:56:10.409 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-11-24T07:56:10.521 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T07:56:10.786 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T07:56:11.180 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T07:56:11.411 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:56:11.538 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:56:11.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:56:11.838 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:56:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:56:12.228 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-24T07:56:12.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T07:56:12.672 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T07:56:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T07:56:13.409 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T07:56:13.636 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:56:13.741 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:56:13.936 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:56:14.082 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:56:14.292 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-24T07:56:14.475 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-24T07:56:14.656 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-24T07:56:14.827 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T07:56:14.972 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T07:56:15.110 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T07:56:15.608 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-11-24T07:56:15.725 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T07:56:15.764 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T07:56:15.943 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T07:56:15.949 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T07:56:16.091 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T07:56:16.230 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T07:56:16.400 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T07:56:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T07:56:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T07:56:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T07:56:17.233 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T07:56:17.415 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T07:56:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-24T07:56:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:56:17.899 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-24T07:56:17.999 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:56:18.111 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T07:56:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:56:18.288 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:56:18.299 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T07:56:18.438 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:56:18.609 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-24T07:56:18.696 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T07:56:18.850 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T07:56:18.979 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T07:56:19.045 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T07:56:19.184 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T07:56:19.440 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T07:56:19.666 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T07:56:19.841 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:56:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:56:20.133 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:56:20.272 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:56:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-24T07:56:20.624 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-24T07:56:20.771 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-24T07:56:20.899 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T07:56:20.949 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T07:56:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T07:56:21.199 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T07:56:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T07:56:22.003 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-24T07:56:22.314 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-24T07:56:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T07:56:22.578 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-24T07:56:22.767 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-24T07:56:22.994 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T07:56:23.004 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T07:56:23.165 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T07:56:23.203 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T07:56:23.381 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T07:56:23.574 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T07:56:23.716 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-24T07:56:23.860 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-24T07:56:23.901 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T07:56:24.197 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-24T07:56:24.208 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T07:56:24.395 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-24T07:56:24.438 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T07:56:24.639 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T07:56:24.675 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-24T07:56:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T07:56:24.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-24T07:56:24.910 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T07:56:25.067 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T07:56:25.166 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T07:56:25.185 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T07:56:25.332 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T07:56:25.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-24T07:56:25.800 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-24T07:56:25.986 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-24T07:56:26.178 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-24T07:56:26.337 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-24T07:56:26.557 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-24T07:56:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-24T07:56:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-24T07:56:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T07:56:27.475 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-24T07:56:27.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-24T07:56:27.947 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-24T07:56:28.188 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-24T07:56:28.320 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-24T07:56:28.484 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-24T07:56:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-24T07:56:28.732 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-24T07:56:28.937 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-24T07:56:28.996 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-24T07:56:29.163 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-24T07:56:29.325 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-24T07:56:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-24T07:56:29.503 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-24T07:56:29.514 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T07:56:29.678 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T07:56:29.833 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-24T07:56:29.872 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T07:56:30.006 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T07:56:30.129 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-24T07:56:30.140 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-24T07:56:30.326 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-24T07:56:30.345 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-24T07:56:30.464 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T07:56:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T07:56:30.576 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-24T07:56:30.740 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T07:56:30.765 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-24T07:56:30.938 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T07:56:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-24T07:56:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T07:56:31.175 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T07:56:31.177 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T07:56:31.374 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-24T07:56:31.390 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T07:56:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-24T07:56:31.562 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T07:56:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-24T07:56:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T07:56:31.856 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-24T07:56:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T07:56:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-24T07:56:32.154 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T07:56:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-24T07:56:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T07:56:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-24T07:56:32.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T07:56:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-24T07:56:32.605 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T07:56:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T07:56:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-24T07:56:33.103 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T07:56:33.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T07:56:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-24T07:56:33.544 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-24T07:56:33.578 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T07:56:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-24T07:56:33.840 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T07:56:34.006 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T07:56:34.105 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-24T07:56:34.209 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T07:56:34.428 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-24T07:56:34.623 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-24T07:56:34.770 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-24T07:56:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-24T07:56:35.154 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-24T07:56:35.311 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T07:56:35.610 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T07:56:35.613 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-24T07:56:35.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T07:56:35.870 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-24T07:56:36.090 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-24T07:56:36.117 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T07:56:36.332 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-24T07:56:36.413 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T07:56:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T07:56:36.609 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T07:56:36.628 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T07:56:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T07:56:37.114 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T07:56:37.261 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T07:56:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T07:56:37.610 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T07:56:37.648 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:56:37.720 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:56:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T07:56:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T07:56:38.011 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T07:56:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T07:56:38.270 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T07:56:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T07:56:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T07:56:38.861 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T07:56:39.109 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T07:56:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T07:56:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T07:56:39.717 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T07:56:40.855 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T07:56:41.290 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T07:56:41.593 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T07:56:41.647 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:56:41.847 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T07:56:41.906 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:56:42.062 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T07:56:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:56:42.781 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:56:42.865 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:56:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T07:56:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:56:43.104 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:56:43.196 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:56:43.355 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T07:56:47.299 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:56:47.504 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:56:48.130 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:56:49.011 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:56:49.095 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:56:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:57:01.659 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:57:01.659 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2023-11-24T07:57:01.659 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2023-11-24T07:57:01.659 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-11-24T07:57:01.660 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:02.028 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:57:03.632 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:57:03.852 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:57:04.085 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:57:04.294 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:57:04.750 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:57:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:57:04.978 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:57:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:57:06.829 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:57:06.959 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T07:57:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T07:57:07.368 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:57:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:57:07.935 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:57:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:57:08.916 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:57:08.916 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2023-11-24T07:57:08.916 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2023-11-24T07:57:08.916 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-11-24T07:57:08.916 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /run 2023-11-24T07:57:08.916 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2023-11-24T07:57:08.917 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2023-11-24T07:57:08.917 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2023-11-24T07:57:08.917 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2023-11-24T07:57:08.917 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-11-24T07:57:08.917 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:09.060 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:57:09.150 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:57:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:57:09.552 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T07:57:09.743 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:57:09.780 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:57:10.001 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:57:10.013 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:57:10.932 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:57:11.116 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:57:11.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T07:57:11.353 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:57:11.485 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:57:11.559 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:57:11.593 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:57:11.810 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:57:11.852 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:57:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:57:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:57:12.143 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:57:12.160 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:57:12.385 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:57:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:57:13.972 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:57:14.169 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:57:14.276 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T07:57:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T07:57:14.617 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:57:14.844 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:57:15.115 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:57:16.139 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:57:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:57:16.389 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:57:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T07:57:16.957 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:57:16.996 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:57:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:57:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:57:17.234 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:57:18.156 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T07:57:18.512 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-24T07:57:18.556 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T07:57:18.778 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:57:18.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 117/134 2023-11-24T07:57:18.813 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:57:18.942 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:57:18.986 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:57:19.026 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-24T07:57:19.125 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:57:19.225 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T07:57:19.415 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:57:19.654 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-24T07:57:19.659 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:57:19.862 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T07:57:20.018 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T07:57:22.175 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:57:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:57:22.435 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:57:22.633 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T07:57:22.916 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T07:57:23.206 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T07:57:23.558 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T07:57:23.732 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T07:57:23.815 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T07:57:24.007 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T07:57:24.020 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T07:57:24.166 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T07:57:24.174 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T07:57:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-24T07:57:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:57:24.485 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:57:24.485 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:57:24.652 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:57:24.720 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:57:24.939 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:57:25.437 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T07:57:25.438 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T07:57:25.439 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T07:57:25.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T07:57:25.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T07:57:25.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T07:57:25.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T07:57:25.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T07:57:25.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T07:57:25.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T07:57:25.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T07:57:25.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T07:57:25.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T07:57:25.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-24T07:57:25.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-24T07:57:25.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-24T07:57:25.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-24T07:57:25.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-24T07:57:25.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T07:57:25.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T07:57:25.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T07:57:25.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:57:25.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T07:57:25.562 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T07:57:25.893 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-24T07:57:26.057 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 117/134 2023-11-24T07:57:26.339 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-24T07:57:26.500 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T07:57:26.707 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T07:57:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-24T07:57:27.104 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T07:57:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T07:57:28.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:57:28.141 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:50 ago on Fri 24 Nov 2023 07:55:38 AM UTC. 2023-11-24T07:57:29.593 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:57:29.832 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:57:29.851 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:57:29.991 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T07:57:30.275 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T07:57:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T07:57:30.681 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-11-24T07:57:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:30.911 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2023-11-24T07:57:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T07:57:30.911 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.912 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.913 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.6-3.el8.x86_64 2023-11-24T07:57:30.914 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:57:30.915 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T07:57:30.916 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:57:30.917 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.918 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T07:57:30.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T07:57:30.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T07:57:30.921 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.922 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T07:57:30.923 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T07:57:30.924 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:30.924 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-11-24T07:57:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T07:57:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T07:57:31.177 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:31.214 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T07:57:31.423 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T07:57:31.437 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T07:57:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T07:57:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T07:57:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-24T07:57:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:57:31.877 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:57:31.877 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:57:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:57:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T07:57:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T07:57:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T07:57:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T07:57:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T07:57:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T07:57:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T07:57:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T07:57:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T07:57:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T07:57:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T07:57:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T07:57:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T07:57:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T07:57:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T07:57:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T07:57:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T07:57:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T07:57:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-24T07:57:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-24T07:57:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-24T07:57:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-24T07:57:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-24T07:57:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-24T07:57:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-24T07:57:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-24T07:57:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-24T07:57:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-24T07:57:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-24T07:57:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-24T07:57:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-24T07:57:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-24T07:57:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-24T07:57:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-24T07:57:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-24T07:57:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-24T07:57:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-24T07:57:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-24T07:57:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-24T07:57:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-24T07:57:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-24T07:57:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T07:57:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T07:57:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:57:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T07:57:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:57:35.500 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:55 ago on Fri 24 Nov 2023 07:55:40 AM UTC. 2023-11-24T07:57:38.590 INFO:teuthology.orchestra.run.smithi032.stdout:Installed products updated. 2023-11-24T07:57:38.909 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:38.909 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2023-11-24T07:57:38.909 INFO:teuthology.orchestra.run.smithi032.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T07:57:38.909 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.909 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.910 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.911 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-3.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T07:57:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T07:57:38.914 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T07:57:38.915 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T07:57:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T07:57:38.917 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T07:57:38.919 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T07:57:38.920 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T07:57:38.921 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T07:57:38.921 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:38.921 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-11-24T07:57:39.233 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:39.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:57:39.380 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T07:57:39.404 INFO:teuthology.orchestra.run.smithi032.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:57:39.405 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:57:39.405 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T07:57:39.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:57:39.624 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T07:57:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:57:39.654 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:57:39.654 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T07:57:39.654 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T07:57:39.655 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:39.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T07:57:39.700 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:39.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T07:57:39.747 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T07:57:39.747 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:39.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T07:57:39.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T07:57:39.844 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:39.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T07:57:39.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T07:57:39.944 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T07:57:39.944 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:39.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T07:57:39.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T07:57:40.041 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:40.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T07:57:40.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T07:57:40.141 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T07:57:40.141 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:40.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T07:57:40.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T07:57:40.235 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:40.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T07:57:40.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T07:57:40.331 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T07:57:40.398 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T07:57:40.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2023-11-24T07:57:40.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2023-11-24T07:57:40.428 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T07:57:40.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T07:57:40.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T07:57:40.495 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T07:57:40.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T07:57:40.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T07:57:40.569 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T07:57:40.569 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T07:57:40.569 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T07:57:40.569 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:40.569 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T07:57:40.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:40.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:40.783 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:40.783 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T07:57:40.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:40.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:40.965 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T07:57:40.965 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T07:57:40.965 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}', 'email': None, 'first_in_suite': False, 'job_id': '7465578', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-tests', 'sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9764, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-tests', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'kclient': None}, {'check-counter': {'counters': {'mgr': [{'min': 4, 'name': 'finisher-volumes.complete_latency.avgcount'}, {'expected_val': 0, 'name': 'finisher-volumes.queue_len'}]}}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestPerModuleFinsherThread']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_07:31:48', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T07:57:40.966 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T07:57:40.968 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T07:57:41.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T07:57:41.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T07:57:41.083 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:41.084 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T07:57:41.111 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T07:57:41.139 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-11-24T07:57:41.168 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 383869 Links: 1 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-11-24 07:57:32.518329101 +0000 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-11-24 07:54:50.889349652 +0000 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-11-24 07:54:50.889349652 +0000 2023-11-24T07:57:41.169 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-11-24T07:57:41.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T07:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-11-24T07:57:41.236 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-11-24T07:57:41.236 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000105655 s, 4.8 MB/s 2023-11-24T07:57:41.237 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T07:57:41.296 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-11-24T07:57:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T07:57:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:41.354 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 387710 Links: 1 2023-11-24T07:57:41.354 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:41.354 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:41.354 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-11-24 07:57:32.519329081 +0000 2023-11-24T07:57:41.355 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-11-24 07:54:51.422338441 +0000 2023-11-24T07:57:41.355 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-11-24 07:54:51.422338441 +0000 2023-11-24T07:57:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-11-24T07:57:41.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T07:57:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-11-24T07:57:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-11-24T07:57:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000113387 s, 4.5 MB/s 2023-11-24T07:57:41.422 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T07:57:41.481 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-11-24T07:57:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T07:57:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:41.539 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 392352 Links: 1 2023-11-24T07:57:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-11-24 07:57:32.519329081 +0000 2023-11-24T07:57:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-11-24 07:54:51.961327103 +0000 2023-11-24T07:57:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-11-24 07:54:51.961327103 +0000 2023-11-24T07:57:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-11-24T07:57:41.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T07:57:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-11-24T07:57:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-11-24T07:57:41.605 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000101697 s, 5.0 MB/s 2023-11-24T07:57:41.606 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T07:57:41.665 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 392502 Links: 1 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-11-24 07:57:32.519329081 +0000 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-11-24 07:54:52.501315745 +0000 2023-11-24T07:57:41.722 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-11-24 07:54:52.501315745 +0000 2023-11-24T07:57:41.723 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-11-24T07:57:41.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T07:57:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-11-24T07:57:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-11-24T07:57:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.3173e-05 s, 5.5 MB/s 2023-11-24T07:57:41.789 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T07:57:41.848 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:41.848 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:41.848 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T07:57:41.867 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T07:57:41.867 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-11-24T07:57:41.926 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T07:57:41.926 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:41.926 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 390376 Links: 1 2023-11-24T07:57:41.926 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:41.926 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:41.927 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-11-24 07:57:25.140107761 +0000 2023-11-24T07:57:41.927 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-11-24 07:54:18.554409213 +0000 2023-11-24T07:57:41.927 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-11-24 07:54:18.554409213 +0000 2023-11-24T07:57:41.927 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-11-24T07:57:41.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T07:57:41.993 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-11-24T07:57:41.993 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-11-24T07:57:41.993 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000261865 s, 2.0 MB/s 2023-11-24T07:57:41.994 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T07:57:42.054 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-11-24T07:57:42.113 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T07:57:42.113 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:42.113 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 392568 Links: 1 2023-11-24T07:57:42.113 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:42.113 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:42.113 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-11-24 07:57:25.141107738 +0000 2023-11-24T07:57:42.114 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-11-24 07:54:19.058397296 +0000 2023-11-24T07:57:42.114 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-11-24 07:54:19.058397296 +0000 2023-11-24T07:57:42.114 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-11-24T07:57:42.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T07:57:42.180 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-11-24T07:57:42.181 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-11-24T07:57:42.181 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000253137 s, 2.0 MB/s 2023-11-24T07:57:42.182 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T07:57:42.241 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 394379 Links: 1 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-11-24 07:57:25.141107738 +0000 2023-11-24T07:57:42.299 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-11-24 07:54:19.566385283 +0000 2023-11-24T07:57:42.300 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-11-24 07:54:19.566385283 +0000 2023-11-24T07:57:42.300 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-11-24T07:57:42.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T07:57:42.365 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-11-24T07:57:42.365 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-11-24T07:57:42.365 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000293751 s, 1.7 MB/s 2023-11-24T07:57:42.366 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T07:57:42.424 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 387796 Links: 1 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-11-24 07:57:25.141107738 +0000 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-11-24 07:54:20.078373177 +0000 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-11-24 07:54:20.078373177 +0000 2023-11-24T07:57:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-11-24T07:57:42.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T07:57:42.546 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-11-24T07:57:42.547 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-11-24T07:57:42.547 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000216152 s, 2.4 MB/s 2023-11-24T07:57:42.548 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T07:57:42.605 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:42.605 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T07:57:42.605 INFO:tasks.ceph:Generating config... 2023-11-24T07:57:42.606 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T07:57:42.606 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T07:57:42.606 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T07:57:42.607 INFO:tasks.ceph:[client] pid file = 2023-11-24T07:57:42.607 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T07:57:42.607 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T07:57:42.607 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T07:57:42.607 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T07:57:42.608 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T07:57:42.609 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T07:57:42.609 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T07:57:42.610 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T07:57:42.610 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T07:57:42.610 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T07:57:42.610 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T07:57:42.610 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T07:57:42.610 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T07:57:42.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T07:57:42.677 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T07:57:42.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T07:57:42.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T07:57:42.761 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.123'), ('mon.c', '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]')] 2023-11-24T07:57:42.761 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T07:57:42.762 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:42.762 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T07:57:42.820 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.123 --addv c '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:42.912 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:42.912 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid 0b3006fa-b594-4439-bcd2-bb19d4a76534 2023-11-24T07:57:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2023-11-24T07:57:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2023-11-24T07:57:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 0b3006fa-b594-4439-bcd2-bb19d4a76534 2023-11-24T07:57:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2023-11-24T07:57:42.911611+0000 2023-11-24T07:57:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-11-24T07:57:42.911611+0000 2023-11-24T07:57:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 16 (pacific) 2023-11-24T07:57:42.914 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2023-11-24T07:57:42.914 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2023-11-24T07:57:42.914 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2023-11-24T07:57:42.914 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] mon.c 2023-11-24T07:57:42.914 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T07:57:42.916 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T07:57:42.934 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0b3006fa-b594-4439-bcd2-bb19d4a76534... 2023-11-24T07:57:42.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T07:57:42.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T07:57:43.022 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2023-11-24T07:57:43.023 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2023-11-24T07:57:43.024 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.025 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2023-11-24T07:57:43.026 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on skipped message = False 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 0b3006fa-b594-4439-bcd2-bb19d4a76534 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.027 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.028 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2023-11-24T07:57:43.029 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.030 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug finisher = 20 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-11-24T07:57:43.031 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.032 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2023-11-24T07:57:43.033 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-11-24T07:57:43.034 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-11-24T07:57:43.035 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-11-24T07:57:43.036 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-11-24T07:57:43.036 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-11-24T07:57:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.038 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-11-24T07:57:43.038 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-11-24T07:57:43.038 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.038 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T07:57:43.038 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.039 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-11-24T07:57:43.040 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-11-24T07:57:43.040 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-11-24T07:57:43.040 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-11-24T07:57:43.040 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-11-24T07:57:43.040 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.041 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.32,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on skipped message = False 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 0b3006fa-b594-4439-bcd2-bb19d4a76534 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-11-24T07:57:43.042 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.043 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-11-24T07:57:43.044 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = bitmap 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 1/20 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 1/20 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-11-24T07:57:43.045 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 4/10 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.046 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-11-24T07:57:43.047 INFO:teuthology.orchestra.run.smithi123.stdout: debug finisher = 20 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-11-24T07:57:43.048 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.049 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-11-24T07:57:43.050 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-11-24T07:57:43.051 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-11-24T07:57:43.052 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-11-24T07:57:43.053 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-11-24T07:57:43.054 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T07:57:43.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T07:57:43.153 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T07:57:43.153 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:43.153 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T07:57:43.170 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:43.170 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T07:57:43.227 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:57:43.227 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:43.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T07:57:43.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T07:57:43.304 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:57:43.304 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:43.361 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:57:43.361 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:43.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T07:57:43.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T07:57:43.407 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:57:43.407 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:43.467 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T07:57:43.467 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T07:57:43.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T07:57:43.522 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T07:57:43.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T07:57:43.649 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T07:57:43.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T07:57:43.705 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T07:57:43.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T07:57:43.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T07:57:43.829 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T07:57:43.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T07:57:43.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T07:57:43.924 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T07:57:43.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T07:57:43.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T07:57:44.052 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T07:57:44.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T07:57:44.080 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T07:57:44.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T07:57:44.127 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T07:57:44.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T07:57:44.186 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T07:57:44.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T07:57:44.243 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T07:57:44.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T07:57:44.298 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T07:57:44.312 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T07:57:44.312 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T07:57:44.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T07:57:44.340 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:44.340 INFO:tasks.ceph:role: osd.0 2023-11-24T07:57:44.340 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:57:44.340 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T07:57:44.407 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:44.408 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:44.430 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-11-24T07:57:44.432 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-11-24T07:57:44.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T07:57:44.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T07:57:44.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T07:57:44.652 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:44.652 INFO:tasks.ceph:role: osd.1 2023-11-24T07:57:44.653 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:57:44.653 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:44.720 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-11-24T07:57:44.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-11-24T07:57:44.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T07:57:44.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T07:57:44.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T07:57:44.961 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:44.962 INFO:tasks.ceph:role: osd.2 2023-11-24T07:57:44.962 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:57:44.962 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:45.028 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:45.029 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:45.051 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-11-24T07:57:45.053 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-11-24T07:57:45.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T07:57:45.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T07:57:45.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T07:57:45.271 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:45.272 INFO:tasks.ceph:role: osd.3 2023-11-24T07:57:45.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:57:45.272 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:45.339 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:45.340 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:45.340 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:45.340 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:45.361 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-11-24T07:57:45.363 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-11-24T07:57:45.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T07:57:45.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T07:57:45.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:45.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:45.616+0000 7fd7cc02a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T07:57:45.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:45.616+0000 7fd7cc02a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T07:57:45.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:45.616+0000 7fd7cc02a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T07:57:45.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:45.616+0000 7fd7cc02a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T07:57:45.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:45.616+0000 7fd7cc02a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T07:57:45.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:45.617+0000 7fd7cc02a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T07:57:48.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T07:57:48.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:48.619+0000 7f2dc373a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T07:57:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:48.619+0000 7f2dc373a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T07:57:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:48.619+0000 7f2dc373a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T07:57:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:48.619+0000 7f2dc373a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T07:57:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:48.619+0000 7f2dc373a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T07:57:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:48.619+0000 7f2dc373a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T07:57:51.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T07:57:51.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:51.644+0000 7f099b317640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T07:57:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:51.644+0000 7f099b317640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T07:57:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:51.644+0000 7f099b317640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T07:57:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:51.644+0000 7f099b317640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T07:57:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:51.644+0000 7f099b317640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T07:57:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:51.644+0000 7f099b317640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T07:57:54.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T07:57:54.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:57:54.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:54.641+0000 7fce611d5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T07:57:54.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:54.641+0000 7fce611d5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T07:57:54.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:54.641+0000 7fce611d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T07:57:54.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:54.641+0000 7fce611d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T07:57:54.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:54.641+0000 7fce611d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T07:57:54.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:57:54.642+0000 7fce611d5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T07:57:57.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T07:57:57.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T07:57:57.574 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:57.574 INFO:tasks.ceph:role: osd.4 2023-11-24T07:57:57.574 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:57:57.574 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T07:57:57.654 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:57.654 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:57.654 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:57.654 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:57.654 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:57.655 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:57.655 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:57.655 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:57.655 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:57.655 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:58.092 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-11-24T07:57:58.094 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-11-24T07:57:58.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T07:57:58.284 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T07:57:58.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T07:57:58.395 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:58.396 INFO:tasks.ceph:role: osd.5 2023-11-24T07:57:58.396 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:57:58.396 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:58.476 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:58.477 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:58.477 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:58.477 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:58.918 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-11-24T07:57:58.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-11-24T07:57:58.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T07:57:59.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T07:57:59.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T07:57:59.159 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:59.159 INFO:tasks.ceph:role: osd.6 2023-11-24T07:57:59.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:57:59.160 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T07:57:59.239 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:57:59.240 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:57:59.683 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-11-24T07:57:59.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-11-24T07:57:59.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T07:57:59.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T07:57:59.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T07:57:59.986 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:57:59.987 INFO:tasks.ceph:role: osd.7 2023-11-24T07:57:59.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:57:59.987 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T07:58:00.066 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:58:00.066 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:58:00.066 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:58:00.067 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:58:00.505 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-11-24T07:58:00.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-11-24T07:58:00.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T07:58:00.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T07:58:00.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:58:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:00.781+0000 7fe07a75f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T07:58:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:00.781+0000 7fe07a75f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T07:58:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:00.781+0000 7fe07a75f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T07:58:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:00.781+0000 7fe07a75f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T07:58:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:00.781+0000 7fe07a75f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T07:58:00.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:00.781+0000 7fe07a75f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T07:58:03.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T07:58:03.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:58:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:03.783+0000 7fbc2650a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T07:58:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:03.783+0000 7fbc2650a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T07:58:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:03.783+0000 7fbc2650a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T07:58:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:03.783+0000 7fbc2650a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T07:58:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:03.783+0000 7fbc2650a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T07:58:03.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:03.783+0000 7fbc2650a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T07:58:06.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T07:58:06.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:58:06.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:06.780+0000 7fc250c54640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T07:58:06.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:06.781+0000 7fc250c54640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T07:58:06.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:06.781+0000 7fc250c54640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T07:58:06.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:06.781+0000 7fc250c54640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T07:58:06.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:06.781+0000 7fc250c54640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T07:58:06.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:06.781+0000 7fc250c54640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T07:58:09.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T07:58:09.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:58:09.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:09.772+0000 7f5c3a94a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T07:58:09.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:09.772+0000 7f5c3a94a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T07:58:09.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:09.772+0000 7f5c3a94a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T07:58:09.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:09.772+0000 7f5c3a94a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T07:58:09.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:09.772+0000 7f5c3a94a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T07:58:09.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:09.772+0000 7f5c3a94a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T07:58:10.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:10.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:12.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T07:58:12.674 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T07:58:12.674 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:12.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T07:58:12.708 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:12.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T07:58:12.781 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:12.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T07:58:12.851 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:12.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T07:58:12.921 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:12.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T07:58:12.990 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:12.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T07:58:13.059 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:13.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T07:58:13.128 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T07:58:13.160 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T07:58:13.233 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T07:58:13.305 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T07:58:13.378 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T07:58:13.451 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.451 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T07:58:13.523 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T07:58:13.596 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:13.596 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T07:58:13.617 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:13.617 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T07:58:13.676 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.676 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T07:58:13.697 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:13.697 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T07:58:13.756 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T07:58:13.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T07:58:13.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB3V2BleKYeHxAA+MBP3PkXhXtzpqBgQjA6Iw== 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB3V2Bl7qkFKhAAWFInAhjEdGK+2n1KAh7cjA== 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB3V2BluWtuMRAALOz2bKOCcqTnzdzCHyTDow== 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB5V2BlVwDPJBAAvXfR6CbEhbJvgvppkmhI9w== 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2023-11-24T07:58:13.800 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB8V2BljdIFJRAARdb3oYMz0SPcwqWAvIJgBA== 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB/V2BlCYR4JhAAKfqAy+WiJUp3Hkfluzow/w== 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCCV2BloCZPJhAA9qI9L0H2XR0yaTZ9Pwq62Q== 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB3V2BlpwCEJhAA13REsSfAK0sL34XS00P0Wg== 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB3V2Bl7AkPNxAAXAxKAeK7bMmmbpkFcrmiyQ== 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB4V2BlF/gTAxAAiEFfMvHjFG1sdfEWlhG6eA== 2023-11-24T07:58:13.801 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCIV2Bl2g+eLhAAxJQVO6RZg05jW5ZWyO9S7A== 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCLV2BlMmG9LhAAtPFiB98UittUzKj9vCx73g== 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCOV2BlgeybLhAAQmviFD27q3NMO6ld08IBAA== 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCRV2BlALMULhAAKTKladOU8UiR0vHK9iwAVg== 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB4V2Bl4D6UBxAADXMYHy5S2dPfRBlHDFLvHg== 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB4V2BlrJAXCxAAOHrSLph/HvP0oOtOiSlTVQ== 2023-11-24T07:58:13.802 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2023-11-24T07:58:13.803 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB4V2BluMN/DhAAMHfkACUnwXufuREwxUli8Q== 2023-11-24T07:58:13.803 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2023-11-24T07:58:13.803 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB4V2BlinzJERAAz0p2a6j4pjcOAI7F6uSxqg== 2023-11-24T07:58:13.827 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2023-11-24T07:58:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB3V2BleKYeHxAA+MBP3PkXhXtzpqBgQjA6Iw== 2023-11-24T07:58:13.827 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-11-24T07:58:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB3V2Bl7qkFKhAAWFInAhjEdGK+2n1KAh7cjA== 2023-11-24T07:58:13.827 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-11-24T07:58:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB3V2BluWtuMRAALOz2bKOCcqTnzdzCHyTDow== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB5V2BlVwDPJBAAvXfR6CbEhbJvgvppkmhI9w== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB8V2BljdIFJRAARdb3oYMz0SPcwqWAvIJgBA== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB/V2BlCYR4JhAAKfqAy+WiJUp3Hkfluzow/w== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCCV2BloCZPJhAA9qI9L0H2XR0yaTZ9Pwq62Q== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB3V2BlpwCEJhAA13REsSfAK0sL34XS00P0Wg== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB3V2Bl7AkPNxAAXAxKAeK7bMmmbpkFcrmiyQ== 2023-11-24T07:58:13.828 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4V2BlF/gTAxAAiEFfMvHjFG1sdfEWlhG6eA== 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCIV2Bl2g+eLhAAxJQVO6RZg05jW5ZWyO9S7A== 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCLV2BlMmG9LhAAtPFiB98UittUzKj9vCx73g== 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCOV2BlgeybLhAAQmviFD27q3NMO6ld08IBAA== 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCRV2BlALMULhAAKTKladOU8UiR0vHK9iwAVg== 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4V2Bl4D6UBxAADXMYHy5S2dPfRBlHDFLvHg== 2023-11-24T07:58:13.829 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2023-11-24T07:58:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4V2BlrJAXCxAAOHrSLph/HvP0oOtOiSlTVQ== 2023-11-24T07:58:13.830 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2023-11-24T07:58:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4V2BluMN/DhAAMHfkACUnwXufuREwxUli8Q== 2023-11-24T07:58:13.830 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2023-11-24T07:58:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4V2BlinzJERAAz0p2a6j4pjcOAI7F6uSxqg== 2023-11-24T07:58:13.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:58:13.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:58:13.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:13.937 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:58:14.290 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:58:14.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:58:14.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:58:14.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:58:14.916 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T07:58:14.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T07:58:14.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T07:58:15.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T07:58:15.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T07:58:15.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T07:58:15.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T07:58:15.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T07:58:15.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T07:58:15.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T07:58:15.409 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:58:15.411 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:58:15.466 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T07:58:15.466 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T07:58:15.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T07:58:15.469 INFO:tasks.ceph.mon.a:Started 2023-11-24T07:58:15.470 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T07:58:15.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T07:58:15.507 INFO:tasks.ceph.mon.b:Started 2023-11-24T07:58:15.508 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T07:58:15.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T07:58:15.510 INFO:tasks.ceph.mon.c:Started 2023-11-24T07:58:15.510 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T07:58:15.510 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T07:58:15.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T07:58:15.512 INFO:tasks.ceph.mgr.y:Started 2023-11-24T07:58:15.512 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T07:58:15.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T07:58:15.514 INFO:tasks.ceph.mgr.x:Started 2023-11-24T07:58:15.514 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T07:58:15.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T07:58:15.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.639+0000 7fa83b37e700 1 Processor -- start 2023-11-24T07:58:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.639+0000 7fa83b37e700 1 -- start start 2023-11-24T07:58:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83b37e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83b37e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83b37e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340b3c60 con 0x7fa8340744d0 2023-11-24T07:58:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83b37e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340b4500 con 0x7fa83415e160 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83b37e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa834074960 con 0x7fa8340afc90 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83991b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa8340744d0 0x7fa83406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57514/0 (socket says 172.21.15.32:57514) 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83991b700 1 -- 172.21.15.32:0/918737180 learned_addr learned my addr 172.21.15.32:0/918737180 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:15.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.640+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T07:58:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:15.841+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T07:58:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T07:58:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T07:58:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:16.242+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa838919700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:17.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:17.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:17.044+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T07:58:18.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 msgr2=0x7fa834073f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8340afc90 0x7fa834073f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 msgr2=0x7fa8340b5da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.123:3300/0 conn(0x7fa83415e160 0x7fa8340b5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 -- 172.21.15.32:0/918737180 >> v1:172.21.15.32:6789/0 conn(0x7fa8340744d0 legacy=0x7fa83406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 -- 172.21.15.32:0/918737180 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340b58c0 con 0x7fa818006220 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa82affd700 1 -- 172.21.15.32:0/918737180 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340b4500 con 0x7fa818003750 2023-11-24T07:58:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.640+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T07:58:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.841+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.841+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.841+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.841+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:18.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:18.841+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T07:58:19.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:19.242+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:19.242+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:19.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:19.242+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:19.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:19.242+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:19.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:19.242+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T07:58:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.043+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.043+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:58:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.043+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:58:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.043+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:58:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.043+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T07:58:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.865+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677403043 0 0) 0x7fa8340b58c0 con 0x7fa818006220 2023-11-24T07:58:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.865+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8200035b0 con 0x7fa818006220 2023-11-24T07:58:20.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971653449 0 0) 0x7fa8200035b0 con 0x7fa818006220 2023-11-24T07:58:20.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 msgr2=0x7fa818005bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:58:20.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 --2- 172.21.15.32:0/918737180 >> v2:172.21.15.32:3300/0 conn(0x7fa818003750 0x7fa818005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81800b000 con 0x7fa818006220 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fa82400e590 con 0x7fa818006220 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 >> v1:172.21.15.123:6789/0 conn(0x7fa818006220 legacy=0x7fa8180086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa83991b700 1 -- 172.21.15.32:0/918737180 reap_dead start 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820003a50 0x7fa820003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa820004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820005410 0x7fa82000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340b4500 con 0x7fa820004850 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82000c2d0 con 0x7fa820005410 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82000c490 con 0x7fa820003a50 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820003a50 0x7fa820003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa820004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.866+0000 7fa83991b700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820005410 0x7fa82000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820005410 msgr2=0x7fa82000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820005410 0x7fa82000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 msgr2=0x7fa820004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa820004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa83911a700 1 -- 172.21.15.32:0/918737180 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200035b0 con 0x7fa820003a50 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa838919700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa820004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa83911a700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820003a50 0x7fa820003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa830019ae0 tx=0x7fa820005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8478ee45ace4c404 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.867+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83001e070 con 0x7fa820003a50 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.868+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000c040 con 0x7fa820003a50 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.868+0000 7fa82bfff700 1 -- 172.21.15.32:0/918737180 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830016ae0 con 0x7fa820003a50 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.868+0000 7fa83b37e700 1 -- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820003a50 msgr2=0x7fa820003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.868+0000 7fa83b37e700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820003a50 0x7fa820003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa830019ae0 tx=0x7fa820005880 comp rx=0 tx=0).stop 2023-11-24T07:58:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.868+0000 7fa83b37e700 1 -- 172.21.15.32:0/918737180 shutdown_connections 2023-11-24T07:58:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.869+0000 7fa83b37e700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820005410 0x7fa82000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.869+0000 7fa83b37e700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa820004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.869+0000 7fa83b37e700 1 --2- 172.21.15.32:0/918737180 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820003a50 0x7fa820003e50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.869+0000 7fa83b37e700 1 -- 172.21.15.32:0/918737180 >> 172.21.15.32:0/918737180 conn(0x7fa8340d0bf0 msgr2=0x7fa8340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.869+0000 7fa83b37e700 1 -- 172.21.15.32:0/918737180 shutdown_connections 2023-11-24T07:58:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.869+0000 7fa83b37e700 1 -- 172.21.15.32:0/918737180 wait complete. 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.871+0000 7fa83b37e700 1 Processor -- start 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.871+0000 7fa83b37e700 1 -- start start 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83b37e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 0x7fa83414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83b37e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa83414e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83b37e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 0x7fa8341f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83b37e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83414b630 con 0x7fa820004850 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83b37e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83414eab0 con 0x7fa820003a50 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83b37e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83414ed80 con 0x7fa820005410 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 0x7fa8341f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83911a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 0x7fa83414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83911a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 0x7fa83414b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40014/0 (socket says 172.21.15.32:40014) 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa838919700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa83414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83911a700 1 -- 172.21.15.32:0/704335399 learned_addr learned my addr 172.21.15.32:0/704335399 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 0x7fa8341f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51120/0 (socket says 172.21.15.32:51120) 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 -- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 msgr2=0x7fa83414b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 0x7fa83414b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 -- 172.21.15.32:0/704335399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 msgr2=0x7fa83414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa83414e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83991b700 1 -- 172.21.15.32:0/704335399 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83414f060 con 0x7fa820005410 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.872+0000 7fa83911a700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 0x7fa83414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.873+0000 7fa83991b700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 0x7fa8341f3130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa83409ef70 tx=0x7fa82c00f1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.875+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00fa20 con 0x7fa820005410 2023-11-24T07:58:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.875+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c00fba0 con 0x7fa820005410 2023-11-24T07:58:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.875+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83414f340 con 0x7fa820005410 2023-11-24T07:58:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.875+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0133f0 con 0x7fa820005410 2023-11-24T07:58:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.876+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83414f900 con 0x7fa820005410 2023-11-24T07:58:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.877+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa834068fb0 con 0x7fa820005410 2023-11-24T07:58:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.877+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa82c013550 con 0x7fa820005410 2023-11-24T07:58:20.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.877+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa82c01d750 con 0x7fa820005410 2023-11-24T07:58:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.881+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa82c013800 con 0x7fa820005410 2023-11-24T07:58:20.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.964+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa83406c140 con 0x7fa820005410 2023-11-24T07:58:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.967+0000 7fa829ffb700 1 -- 172.21.15.32:0/704335399 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa82c02b090 con 0x7fa820005410 2023-11-24T07:58:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.968+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 msgr2=0x7fa8341f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.968+0000 7fa83b37e700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 0x7fa8341f3130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa83409ef70 tx=0x7fa82c00f1d0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 shutdown_connections 2023-11-24T07:58:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa820005410 0x7fa8341f3130 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa820003a50 0x7fa83414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 --2- 172.21.15.32:0/704335399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa820004850 0x7fa83414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 >> 172.21.15.32:0/704335399 conn(0x7fa8340d0bf0 msgr2=0x7fa83415c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 shutdown_connections 2023-11-24T07:58:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:20.969+0000 7fa83b37e700 1 -- 172.21.15.32:0/704335399 wait complete. 2023-11-24T07:58:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2023-11-24T07:58:20.984 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T07:58:20.984 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T07:58:20.984 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:20.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T07:58:20.999 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:20.997+0000 7f82285ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T07:58:21.007 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.005+0000 7f00d0609200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T07:58:21.013 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:21.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T07:58:21.064 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.062+0000 7f82285ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T07:58:21.080 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.078+0000 7f00d0609200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T07:58:21.081 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:21.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T07:58:21.140 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.139+0000 7f82285ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T07:58:21.149 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:21.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T07:58:21.162 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.160+0000 7f00d0609200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T07:58:21.189 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:21.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T07:58:21.216 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:21.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T07:58:21.220 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.218+0000 7f82285ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T07:58:21.245 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:21.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T07:58:21.248 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.247+0000 7f00d0609200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T07:58:21.312 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:21.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T07:58:21.369 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.368+0000 7f82285ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T07:58:21.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new de2c961b-886f-4896-b585-53b827ca14fa 0 2023-11-24T07:58:21.398 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.396+0000 7f00d0609200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T07:58:21.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.537+0000 7f6d23d98700 1 Processor -- start 2023-11-24T07:58:21.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.537+0000 7f6d23d98700 1 -- start start 2023-11-24T07:58:21.547 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d23d98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d1c0b5da0 0x7f6d1c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d23d98700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0b3c60 con 0x7f6d1c15e160 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d23d98700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0b58c0 con 0x7f6d1c0746a0 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d23d98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b4500 con 0x7f6d1c0b5da0 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d21333700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6d1c0746a0 0x7f6d1c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:46922/0 (socket says 172.21.15.123:46922) 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d21333700 1 -- 172.21.15.123:0/695706175 learned_addr learned my addr 172.21.15.123:0/695706175 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d22335700 1 --2- 172.21.15.123:0/695706175 >> v2:172.21.15.32:3300/0 conn(0x7f6d1c0b5da0 0x7f6d1c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507463827 0 0) 0x7f6d1c0b58c0 con 0x7f6d1c0746a0 2023-11-24T07:58:21.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.538+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d080035b0 con 0x7f6d1c0746a0 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956316036 0 0) 0x7f6d080035b0 con 0x7f6d1c0746a0 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 >> v2:172.21.15.32:3300/0 conn(0x7f6d1c0b5da0 msgr2=0x7f6d1c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 --2- 172.21.15.123:0/695706175 >> v2:172.21.15.32:3300/0 conn(0x7f6d1c0b5da0 0x7f6d1c0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 >> v1:172.21.15.32:6789/0 conn(0x7f6d1c15e160 legacy=0x7f6d1c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c06bdc0 con 0x7f6d1c0746a0 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f6d0c001670 con 0x7f6d1c0746a0 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 >> v1:172.21.15.123:6789/0 conn(0x7f6d1c0746a0 legacy=0x7f6d1c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d08003a00 0x7f6d08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d08004400 0x7f6d080048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d080050d0 0x7f6d0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b4500 con 0x7f6d080050d0 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0800c030 con 0x7f6d08003a00 2023-11-24T07:58:21.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0800c1f0 con 0x7f6d08004400 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d08003a00 0x7f6d08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21333700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d08004400 0x7f6d080048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d22335700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d080050d0 0x7f6d0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 -- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d08004400 msgr2=0x7f6d080048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d08004400 0x7f6d080048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 -- 172.21.15.123:0/695706175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d080050d0 msgr2=0x7f6d0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d080050d0 0x7f6d0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d22335700 1 -- 172.21.15.123:0/695706175 reap_dead start 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 -- 172.21.15.123:0/695706175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080035b0 con 0x7f6d08003a00 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21333700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d08004400 0x7f6d080048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:21.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d21b34700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d08003a00 0x7f6d08003de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d18001980 tx=0x7f6d08005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf762a3acfdd457 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18001420 con 0x7f6d08003a00 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d180052d0 con 0x7f6d08003a00 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.539+0000 7f6d20b32700 1 -- 172.21.15.123:0/695706175 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18003680 con 0x7f6d08003a00 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 -- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d08003a00 msgr2=0x7f6d08003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d08003a00 0x7f6d08003de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d18001980 tx=0x7f6d08005880 comp rx=0 tx=0).stop 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 -- 172.21.15.123:0/695706175 shutdown_connections 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 --2- 172.21.15.123:0/695706175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d08003a00 0x7f6d08003de0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 -- 172.21.15.123:0/695706175 >> 172.21.15.123:0/695706175 conn(0x7f6d1c0d0bf0 msgr2=0x7f6d1c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 -- 172.21.15.123:0/695706175 shutdown_connections 2023-11-24T07:58:21.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.540+0000 7f6d23d98700 1 -- 172.21.15.123:0/695706175 wait complete. 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.542+0000 7f6d23d98700 1 Processor -- start 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.542+0000 7f6d23d98700 1 -- start start 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d08003a00 0x7f6d1c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d1c1547f0 0x7f6d1c1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 0x7f6d1c0b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b0f50 con 0x7f6d08003a00 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c159d90 con 0x7f6d1c1547f0 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c154fd0 con 0x7f6d1c0b0590 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 0x7f6d1c0b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d21333700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d1c1547f0 0x7f6d1c1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 0x7f6d1c0b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:51148/0 (socket says 172.21.15.123:51148) 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d21333700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d1c1547f0 0x7f6d1c1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56654/0 (socket says 172.21.15.123:56654) 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 -- 172.21.15.123:0/2153364602 learned_addr learned my addr 172.21.15.123:0/2153364602 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d21b34700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d08003a00 0x7f6d1c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 -- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d1c1547f0 msgr2=0x7f6d1c1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d1c1547f0 0x7f6d1c1445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 -- 172.21.15.123:0/2153364602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d08003a00 msgr2=0x7f6d1c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d08003a00 0x7f6d1c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 -- 172.21.15.123:0/2153364602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c159740 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d22335700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 0x7f6d1c0b0a10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c075050 tx=0x7f6d14011ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400e040 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1400f040 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c159a20 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400fc40 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.543+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c1552b0 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.544+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6d14022760 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.544+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6d14015700 con 0x7f6d1c0b0590 2023-11-24T07:58:21.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.544+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c15a070 con 0x7f6d1c0b0590 2023-11-24T07:58:21.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.548+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6d14015910 con 0x7f6d1c0b0590 2023-11-24T07:58:21.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.633+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "de2c961b-886f-4896-b585-53b827ca14fa", "id": 0} v 0) v1 -- 0x7f6d1c159ed0 con 0x7f6d1c0b0590 2023-11-24T07:58:21.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.636+0000 7f6d127fc700 1 -- 172.21.15.123:0/2153364602 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de2c961b-886f-4896-b585-53b827ca14fa", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6d1c159ed0 con 0x7f6d1c0b0590 2023-11-24T07:58:21.638 INFO:teuthology.orchestra.run.smithi123.stdout:0 2023-11-24T07:58:21.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 msgr2=0x7f6d1c0b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 0x7f6d1c0b0a10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c075050 tx=0x7f6d14011ed0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 shutdown_connections 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d1c0b0590 0x7f6d1c0b0a10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d08003a00 0x7f6d1c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 --2- 172.21.15.123:0/2153364602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d1c1547f0 0x7f6d1c1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 >> 172.21.15.123:0/2153364602 conn(0x7f6d1c0d0bf0 msgr2=0x7f6d1c0cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 shutdown_connections 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.637+0000 7f6d23d98700 1 -- 172.21.15.123:0/2153364602 wait complete. 2023-11-24T07:58:21.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new c549a8a2-fb37-4671-874e-5f3bfe42a83d 1 2023-11-24T07:58:21.770 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.769+0000 7f4fc1ef8700 1 Processor -- start 2023-11-24T07:58:21.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.769+0000 7f4fc1ef8700 1 -- start start 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fc1ef8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4fbc0ccae0 0x7f4fbc0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fc1ef8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4fbc0cb470 0x7f4fbc0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fc1ef8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc0c5b10 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fc1ef8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc06ee40 con 0x7f4fbc0ccae0 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fc1ef8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c60d0 con 0x7f4fbc0cb470 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbaffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4fbc0cb470 0x7f4fbc0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbbfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4fbc0cbe10 0x7f4fbc06d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:46942/0 (socket says 172.21.15.123:46942) 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbbfff700 1 -- 172.21.15.123:0/2136734468 learned_addr learned my addr 172.21.15.123:0/2136734468 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbb7fe700 1 --2- 172.21.15.123:0/2136734468 >> v2:172.21.15.32:3300/0 conn(0x7f4fbc0ccae0 0x7f4fbc0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbaffd700 1 -- 172.21.15.123:0/2136734468 >> v2:172.21.15.32:3300/0 conn(0x7f4fbc0ccae0 msgr2=0x7f4fbc0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbaffd700 1 --2- 172.21.15.123:0/2136734468 >> v2:172.21.15.32:3300/0 conn(0x7f4fbc0ccae0 0x7f4fbc0ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbaffd700 1 -- 172.21.15.123:0/2136734468 >> v1:172.21.15.123:6789/0 conn(0x7f4fbc0cbe10 legacy=0x7f4fbc06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbaffd700 1 -- 172.21.15.123:0/2136734468 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0cd790 con 0x7f4fbc0cb470 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.770+0000 7f4fbaffd700 1 --2- 172.21.15.123:0/2136734468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc0cb8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc06cd20 tx=0x7f4fb0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896bb9c55935da72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fba7fc700 1 -- 172.21.15.123:0/2136734468 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0010a90 con 0x7f4fbc0cb470 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fba7fc700 1 -- 172.21.15.123:0/2136734468 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0010bf0 con 0x7f4fbc0cb470 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fba7fc700 1 -- 172.21.15.123:0/2136734468 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0018440 con 0x7f4fbc0cb470 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/2136734468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cb470 msgr2=0x7f4fbc0cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/2136734468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc0cb8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc06cd20 tx=0x7f4fb0009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/2136734468 shutdown_connections 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/2136734468 >> v2:172.21.15.32:3300/0 conn(0x7f4fbc0ccae0 0x7f4fbc0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/2136734468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc0cb8d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/2136734468 >> 172.21.15.123:0/2136734468 conn(0x7f4fbc0726e0 msgr2=0x7f4fbc072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/2136734468 shutdown_connections 2023-11-24T07:58:21.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.771+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/2136734468 wait complete. 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.772+0000 7f4fc1ef8700 1 Processor -- start 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 -- start start 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 0x7f4fbc0c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4fbc0ccae0 0x7f4fbc151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c8ed0 con 0x7f4fbc0cb470 2023-11-24T07:58:21.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c91b0 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fc1ef8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c9490 con 0x7f4fbc0ccae0 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fbaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 0x7f4fbc0c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.773+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4fbc0ccae0 0x7f4fbc151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 0x7f4fbc0c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56666/0 (socket says 172.21.15.123:56666) 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 -- 172.21.15.123:0/3970988602 learned_addr learned my addr 172.21.15.123:0/3970988602 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbb7fe700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 -- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4fbc0ccae0 msgr2=0x7f4fbc151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4fbc0ccae0 0x7f4fbc151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 -- 172.21.15.123:0/3970988602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fbc0cb470 msgr2=0x7f4fbc157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 -- 172.21.15.123:0/3970988602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb00094a0 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbb7fe700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbbfff700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4fbc0ccae0 0x7f4fbc151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fbaffd700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 0x7f4fbc0c8530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0000c00 tx=0x7f4fb0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003bf0 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0003d70 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0023690 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc0c9770 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc0c9f70 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4fb001f070 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.774+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f4fb0018880 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.780 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.775+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc068fb0 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.780 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.777+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4fb004ace0 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.786 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.785+0000 7f82285ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T07:58:21.808 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.807+0000 7f00d0609200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T07:58:21.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.862+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c549a8a2-fb37-4671-874e-5f3bfe42a83d", "id": 1} v 0) v1 -- 0x7f4fbc0c58f0 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.866+0000 7f4fa3fff700 1 -- 172.21.15.123:0/3970988602 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c549a8a2-fb37-4671-874e-5f3bfe42a83d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4fb0039070 con 0x7f4fbc0cbe10 2023-11-24T07:58:21.868 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-11-24T07:58:21.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 msgr2=0x7f4fbc0c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 0x7f4fbc0c8530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0000c00 tx=0x7f4fb0004750 comp rx=0 tx=0).stop 2023-11-24T07:58:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 shutdown_connections 2023-11-24T07:58:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4fbc0ccae0 0x7f4fbc151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fbc0cb470 0x7f4fbc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 --2- 172.21.15.123:0/3970988602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc0cbe10 0x7f4fbc0c8530 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.867+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 >> 172.21.15.123:0/3970988602 conn(0x7f4fbc0726e0 msgr2=0x7f4fbc0af220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.868+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 shutdown_connections 2023-11-24T07:58:21.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:21.868+0000 7f4fc1ef8700 1 -- 172.21.15.123:0/3970988602 wait complete. 2023-11-24T07:58:21.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new a71282b6-7723-43e7-aab3-5a78e9fd15ce 2 2023-11-24T07:58:21.908 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.907+0000 7f82285ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T07:58:21.929 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:21.928+0000 7f00d0609200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T07:58:22.000 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:21.998+0000 7f82285ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T07:58:22.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.002+0000 7feaa9ddd700 1 Processor -- start 2023-11-24T07:58:22.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.002+0000 7feaa9ddd700 1 -- start start 2023-11-24T07:58:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa9ddd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa415e160 0x7feaa40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa9ddd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 0x7feaa406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa9ddd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa40b3c60 con 0x7feaa40afc90 2023-11-24T07:58:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa9ddd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa40b4500 con 0x7feaa40746a0 2023-11-24T07:58:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa9ddd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40b58c0 con 0x7feaa415e160 2023-11-24T07:58:22.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 0x7feaa406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa415e160 0x7feaa40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa2ffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feaa40afc90 0x7feaa4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:46952/0 (socket says 172.21.15.123:46952) 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa2ffd700 1 -- 172.21.15.123:0/692527006 learned_addr learned my addr 172.21.15.123:0/692527006 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa3fff700 1 --2- 172.21.15.123:0/692527006 >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 0x7feaa406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56674/0 (socket says 172.21.15.123:56674) 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa37fe700 1 -- 172.21.15.123:0/692527006 >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 msgr2=0x7feaa406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa37fe700 1 --2- 172.21.15.123:0/692527006 >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 0x7feaa406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa37fe700 1 -- 172.21.15.123:0/692527006 >> v1:172.21.15.123:6789/0 conn(0x7feaa40afc90 legacy=0x7feaa4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa37fe700 1 -- 172.21.15.123:0/692527006 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa406bdc0 con 0x7feaa415e160 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa27fc700 1 -- 172.21.15.123:0/692527006 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709655109 0 0) 0x7feaa40b3c60 con 0x7feaa40afc90 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.003+0000 7feaa3fff700 1 --2- 172.21.15.123:0/692527006 >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 0x7feaa406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa37fe700 1 --2- 172.21.15.123:0/692527006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa415e160 0x7feaa40b5da0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea94009840 tx=0x7fea94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5e4fb18a0b0d5f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa27fc700 1 -- 172.21.15.123:0/692527006 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94011830 con 0x7feaa415e160 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa27fc700 1 -- 172.21.15.123:0/692527006 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea94011990 con 0x7feaa415e160 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa27fc700 1 -- 172.21.15.123:0/692527006 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94011d20 con 0x7feaa415e160 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 -- 172.21.15.123:0/692527006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa415e160 msgr2=0x7feaa40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/692527006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa415e160 0x7feaa40b5da0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea94009840 tx=0x7fea94009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 -- 172.21.15.123:0/692527006 shutdown_connections 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/692527006 >> v2:172.21.15.123:3300/0 conn(0x7feaa40746a0 0x7feaa406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/692527006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa415e160 0x7feaa40b5da0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 -- 172.21.15.123:0/692527006 >> 172.21.15.123:0/692527006 conn(0x7feaa40d0bf0 msgr2=0x7feaa40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 -- 172.21.15.123:0/692527006 shutdown_connections 2023-11-24T07:58:22.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.004+0000 7feaa9ddd700 1 -- 172.21.15.123:0/692527006 wait complete. 2023-11-24T07:58:22.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 Processor -- start 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 -- start start 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 0x7feaa41f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa40afc90 0x7feaa40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa415e160 0x7feaa40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c6110 con 0x7feaa415e160 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c6300 con 0x7feaa40746a0 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.006+0000 7feaa9ddd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c65e0 con 0x7feaa40afc90 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 0x7feaa41f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 0x7feaa41f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56676/0 (socket says 172.21.15.123:56676) 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa2ffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa40afc90 0x7feaa40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 -- 172.21.15.123:0/3596009744 learned_addr learned my addr 172.21.15.123:0/3596009744 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:22.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa3fff700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa415e160 0x7feaa40cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 -- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa40afc90 msgr2=0x7feaa40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa40afc90 0x7feaa40c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 -- 172.21.15.123:0/3596009744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa415e160 msgr2=0x7feaa40cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa415e160 0x7feaa40cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 -- 172.21.15.123:0/3596009744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c009a70 con 0x7feaa40746a0 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa3fff700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa415e160 0x7feaa40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa2ffd700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa40afc90 0x7feaa40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa37fe700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 0x7feaa41f4600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea940090b0 tx=0x7fea94004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940036a0 con 0x7feaa40746a0 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9401f070 con 0x7feaa40746a0 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea940094a0 con 0x7feaa40746a0 2023-11-24T07:58:22.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94003a00 con 0x7feaa40746a0 2023-11-24T07:58:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.007+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa40c6ba0 con 0x7feaa40746a0 2023-11-24T07:58:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.008+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fea94026480 con 0x7feaa40746a0 2023-11-24T07:58:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.008+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fea94021070 con 0x7feaa40746a0 2023-11-24T07:58:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.008+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa4068fb0 con 0x7feaa40746a0 2023-11-24T07:58:22.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.010+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fea9402b020 con 0x7feaa40746a0 2023-11-24T07:58:22.022 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:22.021+0000 7f00d0609200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T07:58:22.085 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:22.083+0000 7f82285ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T07:58:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.096+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a71282b6-7723-43e7-aab3-5a78e9fd15ce", "id": 2} v 0) v1 -- 0x7feaa40b21e0 con 0x7feaa40746a0 2023-11-24T07:58:22.100 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.099+0000 7fea8bfff700 1 -- 172.21.15.123:0/3596009744 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a71282b6-7723-43e7-aab3-5a78e9fd15ce", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fea94015020 con 0x7feaa40746a0 2023-11-24T07:58:22.101 INFO:teuthology.orchestra.run.smithi123.stdout:2 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 msgr2=0x7feaa41f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 0x7feaa41f4600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea940090b0 tx=0x7fea94004750 comp rx=0 tx=0).stop 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 shutdown_connections 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feaa415e160 0x7feaa40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa40746a0 0x7feaa41f4600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 --2- 172.21.15.123:0/3596009744 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaa40afc90 0x7feaa40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 >> 172.21.15.123:0/3596009744 conn(0x7feaa40d0bf0 msgr2=0x7feaa415d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 shutdown_connections 2023-11-24T07:58:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.100+0000 7feaa9ddd700 1 -- 172.21.15.123:0/3596009744 wait complete. 2023-11-24T07:58:22.107 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:22.106+0000 7f00d0609200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T07:58:22.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new e6eee010-f6ae-485d-b7d7-7ea8d03473d6 3 2023-11-24T07:58:22.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.275+0000 7f08d43db700 1 Processor -- start 2023-11-24T07:58:22.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.275+0000 7f08d43db700 1 -- start start 2023-11-24T07:58:22.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d43db700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f08cc0c6c30 0x7f08cc0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d43db700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08cc0cef40 0x7f08cc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d43db700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08cc0d0140 con 0x7f08cc15bf50 2023-11-24T07:58:22.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d43db700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc15c330 con 0x7f08cc0cef40 2023-11-24T07:58:22.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d43db700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc0cf3d0 con 0x7f08cc0c6c30 2023-11-24T07:58:22.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d1976700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f08cc0c6c30 0x7f08cc0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d2177700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f08cc15bf50 0x7f08cc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:46962/0 (socket says 172.21.15.123:46962) 2023-11-24T07:58:22.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.276+0000 7f08d2177700 1 -- 172.21.15.123:0/4032777502 learned_addr learned my addr 172.21.15.123:0/4032777502 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-11-24T07:58:22.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d2978700 1 --2- 172.21.15.123:0/4032777502 >> v2:172.21.15.32:3300/0 conn(0x7f08cc0cef40 0x7f08cc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1976700 1 -- 172.21.15.123:0/4032777502 >> v2:172.21.15.32:3300/0 conn(0x7f08cc0cef40 msgr2=0x7f08cc0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1976700 1 --2- 172.21.15.123:0/4032777502 >> v2:172.21.15.32:3300/0 conn(0x7f08cc0cef40 0x7f08cc0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1976700 1 -- 172.21.15.123:0/4032777502 >> v1:172.21.15.123:6789/0 conn(0x7f08cc15bf50 legacy=0x7f08cc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1976700 1 -- 172.21.15.123:0/4032777502 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0d03e0 con 0x7f08cc0c6c30 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1976700 1 --2- 172.21.15.123:0/4032777502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0c6c30 0x7f08cc0cea00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f08cc072420 tx=0x7f08bc009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f2f5f04e152c0bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1175700 1 -- 172.21.15.123:0/4032777502 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc011830 con 0x7f08cc0c6c30 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1175700 1 -- 172.21.15.123:0/4032777502 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc011990 con 0x7f08cc0c6c30 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.277+0000 7f08d1175700 1 -- 172.21.15.123:0/4032777502 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc011d20 con 0x7f08cc0c6c30 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 -- 172.21.15.123:0/4032777502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0c6c30 msgr2=0x7f08cc0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 --2- 172.21.15.123:0/4032777502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0c6c30 0x7f08cc0cea00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f08cc072420 tx=0x7f08bc009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 -- 172.21.15.123:0/4032777502 shutdown_connections 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 --2- 172.21.15.123:0/4032777502 >> v2:172.21.15.32:3300/0 conn(0x7f08cc0cef40 0x7f08cc0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 --2- 172.21.15.123:0/4032777502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0c6c30 0x7f08cc0cea00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 -- 172.21.15.123:0/4032777502 >> 172.21.15.123:0/4032777502 conn(0x7f08cc0b3780 msgr2=0x7f08cc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 -- 172.21.15.123:0/4032777502 shutdown_connections 2023-11-24T07:58:22.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.278+0000 7f08d43db700 1 -- 172.21.15.123:0/4032777502 wait complete. 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.279+0000 7f08d43db700 1 Processor -- start 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.279+0000 7f08d43db700 1 -- start start 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d43db700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc0c6c30 0x7f08cc1f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d43db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0cef40 0x7f08cc0c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d43db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 0x7f08cc0c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d43db700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0b5320 con 0x7f08cc0cef40 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d43db700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0b49f0 con 0x7f08cc15bf50 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d43db700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0b4cd0 con 0x7f08cc0c6c30 2023-11-24T07:58:22.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d2978700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 0x7f08cc0c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d2177700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc0c6c30 0x7f08cc1f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d2978700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 0x7f08cc0c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56692/0 (socket says 172.21.15.123:56692) 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d2978700 1 -- 172.21.15.123:0/870829157 learned_addr learned my addr 172.21.15.123:0/870829157 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.280+0000 7f08d1976700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0cef40 0x7f08cc0c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d2978700 1 -- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc0c6c30 msgr2=0x7f08cc1f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d2978700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc0c6c30 0x7f08cc1f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d2978700 1 -- 172.21.15.123:0/870829157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0cef40 msgr2=0x7f08cc0c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d2978700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0cef40 0x7f08cc0c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d2978700 1 -- 172.21.15.123:0/870829157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc0094a0 con 0x7f08cc15bf50 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d2978700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 0x7f08cc0c7d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08c400efc0 tx=0x7f08c400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4009ea0 con 0x7f08cc15bf50 2023-11-24T07:58:22.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c400f040 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc0b4fb0 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400ced0 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc1f17c0 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f08c4014780 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.281+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f08c4013870 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.282+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc0b5490 con 0x7f08cc15bf50 2023-11-24T07:58:22.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.284+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f08c4018020 con 0x7f08cc15bf50 2023-11-24T07:58:22.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.369+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6eee010-f6ae-485d-b7d7-7ea8d03473d6", "id": 3} v 0) v1 -- 0x7f08cc068fb0 con 0x7f08cc15bf50 2023-11-24T07:58:22.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.377+0000 7f08c2ffd700 1 -- 172.21.15.123:0/870829157 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6eee010-f6ae-485d-b7d7-7ea8d03473d6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f08c4014c60 con 0x7f08cc15bf50 2023-11-24T07:58:22.379 INFO:teuthology.orchestra.run.smithi123.stdout:3 2023-11-24T07:58:22.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 msgr2=0x7f08cc0c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 0x7f08cc0c7d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08c400efc0 tx=0x7f08c400c3a0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 shutdown_connections 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc15bf50 0x7f08cc0c7d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc0c6c30 0x7f08cc1f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 --2- 172.21.15.123:0/870829157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0cef40 0x7f08cc0c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 >> 172.21.15.123:0/870829157 conn(0x7f08cc0b3780 msgr2=0x7f08cc06f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 shutdown_connections 2023-11-24T07:58:22.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.378+0000 7f08d43db700 1 -- 172.21.15.123:0/870829157 wait complete. 2023-11-24T07:58:22.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new eba71c79-ee64-4d18-9f0e-ead114080731 4 2023-11-24T07:58:22.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.511+0000 7f3d59e56700 1 Processor -- start 2023-11-24T07:58:22.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.511+0000 7f3d59e56700 1 -- start start 2023-11-24T07:58:22.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d59e56700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 0x7f3d540d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d59e56700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d540b93c0 0x7f3d5406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d59e56700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d540b6fb0 con 0x7f3d540b3250 2023-11-24T07:58:22.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d59e56700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d540d4da0 con 0x7f3d540b93c0 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d59e56700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540d2940 con 0x7f3d5415e320 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d4bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d540b93c0 0x7f3d5406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d537fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 0x7f3d540d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d4bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d540b93c0 0x7f3d5406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56694/0 (socket says 172.21.15.123:56694) 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d537fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 0x7f3d540d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:51186/0 (socket says 172.21.15.123:51186) 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d4bfff700 1 -- 172.21.15.123:0/3354911298 learned_addr learned my addr 172.21.15.123:0/3354911298 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d4bfff700 1 -- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 msgr2=0x7f3d540d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d4bfff700 1 --2- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 0x7f3d540d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d4bfff700 1 -- 172.21.15.123:0/3354911298 >> v1:172.21.15.123:6789/0 conn(0x7f3d540b3250 legacy=0x7f3d540ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d4bfff700 1 -- 172.21.15.123:0/3354911298 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d540d2c10 con 0x7f3d540b93c0 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.512+0000 7f3d52ffd700 1 -- 172.21.15.123:0/3354911298 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191053646 0 0) 0x7f3d540b6fb0 con 0x7f3d540b3250 2023-11-24T07:58:22.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d537fe700 1 --2- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 0x7f3d540d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d4bfff700 1 --2- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b93c0 0x7f3d5406aff0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098f0 tx=0x7f3d3c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f555d2bc55a92dc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d52ffd700 1 -- 172.21.15.123:0/3354911298 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011830 con 0x7f3d540b93c0 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d52ffd700 1 -- 172.21.15.123:0/3354911298 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c011990 con 0x7f3d540b93c0 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d52ffd700 1 -- 172.21.15.123:0/3354911298 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c011d20 con 0x7f3d540b93c0 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d59e56700 1 -- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b93c0 msgr2=0x7f3d5406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.513+0000 7f3d59e56700 1 --2- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b93c0 0x7f3d5406aff0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098f0 tx=0x7f3d3c009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.514+0000 7f3d59e56700 1 -- 172.21.15.123:0/3354911298 shutdown_connections 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.514+0000 7f3d59e56700 1 --2- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d5415e320 0x7f3d540d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.514+0000 7f3d59e56700 1 --2- 172.21.15.123:0/3354911298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b93c0 0x7f3d5406aff0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.514+0000 7f3d59e56700 1 -- 172.21.15.123:0/3354911298 >> 172.21.15.123:0/3354911298 conn(0x7f3d540d3e50 msgr2=0x7f3d540d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.514+0000 7f3d59e56700 1 -- 172.21.15.123:0/3354911298 shutdown_connections 2023-11-24T07:58:22.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.514+0000 7f3d59e56700 1 -- 172.21.15.123:0/3354911298 wait complete. 2023-11-24T07:58:22.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.515+0000 7f3d59e56700 1 Processor -- start 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.515+0000 7f3d59e56700 1 -- start start 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d59e56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 0x7f3d540aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d59e56700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d540b93c0 0x7f3d541f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d59e56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5415e320 0x7f3d541f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d59e56700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540cb8f0 con 0x7f3d5415e320 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d59e56700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540aae50 con 0x7f3d540b3250 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d59e56700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540cafc0 con 0x7f3d540b93c0 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 0x7f3d540aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d540b93c0 0x7f3d541f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 0x7f3d540aa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56698/0 (socket says 172.21.15.123:56698) 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.516+0000 7f3d537fe700 1 -- 172.21.15.123:0/2135095187 learned_addr learned my addr 172.21.15.123:0/2135095187 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d53fff700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5415e320 0x7f3d541f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d4bfff700 1 -- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 msgr2=0x7f3d540aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d4bfff700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 0x7f3d540aa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d4bfff700 1 -- 172.21.15.123:0/2135095187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5415e320 msgr2=0x7f3d541f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d4bfff700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5415e320 0x7f3d541f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d4bfff700 1 -- 172.21.15.123:0/2135095187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440094a0 con 0x7f3d540b93c0 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d537fe700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 0x7f3d540aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d53fff700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5415e320 0x7f3d541f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d4bfff700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d540b93c0 0x7f3d541f4830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c016000 tx=0x7f3d3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c004400 con 0x7f3d540b93c0 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c01f070 con 0x7f3d540b93c0 2023-11-24T07:58:22.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d59e56700 1 -- 172.21.15.123:0/2135095187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c009550 con 0x7f3d540b93c0 2023-11-24T07:58:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0038c0 con 0x7f3d540b93c0 2023-11-24T07:58:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d59e56700 1 -- 172.21.15.123:0/2135095187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d540cb580 con 0x7f3d540b93c0 2023-11-24T07:58:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3d3c021030 con 0x7f3d540b93c0 2023-11-24T07:58:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.517+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3d3c01ab00 con 0x7f3d540b93c0 2023-11-24T07:58:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.518+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c0052e0 con 0x7f3d540b93c0 2023-11-24T07:58:22.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.520+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f3d3c015020 con 0x7f3d540b93c0 2023-11-24T07:58:22.540 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:22.538+0000 7f82285ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T07:58:22.554 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:22.553+0000 7f00d0609200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T07:58:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.604+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "eba71c79-ee64-4d18-9f0e-ead114080731", "id": 4} v 0) v1 -- 0x7f3d2c005900 con 0x7f3d540b93c0 2023-11-24T07:58:22.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.611+0000 7f3d50ff9700 1 -- 172.21.15.123:0/2135095187 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eba71c79-ee64-4d18-9f0e-ead114080731", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d3c01ad10 con 0x7f3d540b93c0 2023-11-24T07:58:22.613 INFO:teuthology.orchestra.run.smithi123.stdout:4 2023-11-24T07:58:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d540b93c0 msgr2=0x7f3d541f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d540b93c0 0x7f3d541f4830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c016000 tx=0x7f3d3c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 shutdown_connections 2023-11-24T07:58:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5415e320 0x7f3d541f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d540b3250 0x7f3d540aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 --2- 172.21.15.123:0/2135095187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d540b93c0 0x7f3d541f4830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 >> 172.21.15.123:0/2135095187 conn(0x7f3d540d3e50 msgr2=0x7f3d540b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 shutdown_connections 2023-11-24T07:58:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.612+0000 7f3d49ffb700 1 -- 172.21.15.123:0/2135095187 wait complete. 2023-11-24T07:58:22.626 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 746138ca-9c6f-4895-a296-bd6778103fed 5 2023-11-24T07:58:22.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.744+0000 7f9cd739e700 1 Processor -- start 2023-11-24T07:58:22.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.744+0000 7f9cd739e700 1 -- start start 2023-11-24T07:58:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd739e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cd0158310 0x7f9cd00ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd739e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cd006c710 0x7f9cd00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd739e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd01586f0 con 0x7f9cd00addb0 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd739e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00ae2c0 con 0x7f9cd0158310 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd739e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd015e3c0 con 0x7f9cd006c710 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cd006c710 0x7f9cd00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cd006c710 0x7f9cd00b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56708/0 (socket says 172.21.15.123:56708) 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 -- 172.21.15.123:0/2956306589 learned_addr learned my addr 172.21.15.123:0/2956306589 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd639c700 1 --2- 172.21.15.123:0/2956306589 >> v2:172.21.15.32:3300/0 conn(0x7f9cd0158310 0x7f9cd00ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 -- 172.21.15.123:0/2956306589 >> v2:172.21.15.32:3300/0 conn(0x7f9cd0158310 msgr2=0x7f9cd00ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 --2- 172.21.15.123:0/2956306589 >> v2:172.21.15.32:3300/0 conn(0x7f9cd0158310 0x7f9cd00ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 -- 172.21.15.123:0/2956306589 >> v1:172.21.15.32:6789/0 conn(0x7f9cd00addb0 legacy=0x7f9cd006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 -- 172.21.15.123:0/2956306589 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd015e6a0 con 0x7f9cd006c710 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.745+0000 7f9cd6b9d700 1 --2- 172.21.15.123:0/2956306589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd00b3d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0073070 tx=0x7f9cc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c70793ca68f5c96 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd539a700 1 -- 172.21.15.123:0/2956306589 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800e070 con 0x7f9cd006c710 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd539a700 1 -- 172.21.15.123:0/2956306589 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc800b910 con 0x7f9cd006c710 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd539a700 1 -- 172.21.15.123:0/2956306589 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800bcc0 con 0x7f9cd006c710 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 -- 172.21.15.123:0/2956306589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 msgr2=0x7f9cd00b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 --2- 172.21.15.123:0/2956306589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd00b3d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0073070 tx=0x7f9cc800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 -- 172.21.15.123:0/2956306589 shutdown_connections 2023-11-24T07:58:22.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 --2- 172.21.15.123:0/2956306589 >> v2:172.21.15.32:3300/0 conn(0x7f9cd0158310 0x7f9cd00ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 --2- 172.21.15.123:0/2956306589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd00b3d00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 -- 172.21.15.123:0/2956306589 >> 172.21.15.123:0/2956306589 conn(0x7f9cd0071870 msgr2=0x7f9cd0071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 -- 172.21.15.123:0/2956306589 shutdown_connections 2023-11-24T07:58:22.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.746+0000 7f9cd739e700 1 -- 172.21.15.123:0/2956306589 wait complete. 2023-11-24T07:58:22.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 Processor -- start 2023-11-24T07:58:22.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 -- start start 2023-11-24T07:58:22.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd01428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd00addb0 0x7f9cd00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cd0158310 0x7f9cd01e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0142e00 con 0x7f9cd00addb0 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0143090 con 0x7f9cd006c710 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.748+0000 7f9cd739e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0143370 con 0x7f9cd0158310 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd01428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd01428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56710/0 (socket says 172.21.15.123:56710) 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 -- 172.21.15.123:0/3052466357 learned_addr learned my addr 172.21.15.123:0/3052466357 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd5b9b700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd00addb0 0x7f9cd00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd6b9d700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cd0158310 0x7f9cd01e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 -- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cd0158310 msgr2=0x7f9cd01e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cd0158310 0x7f9cd01e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 -- 172.21.15.123:0/3052466357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd00addb0 msgr2=0x7f9cd00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd00addb0 0x7f9cd00ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 -- 172.21.15.123:0/3052466357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8009b30 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd639c700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd01428c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e870 tx=0x7f9ccc00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc009590 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc013680 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd0143650 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0139f0 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd0143c10 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9ccc005a20 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.749+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9ccc0066b0 con 0x7f9cd006c710 2023-11-24T07:58:22.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.750+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd00751e0 con 0x7f9cd006c710 2023-11-24T07:58:22.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.752+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9ccc021060 con 0x7f9cd006c710 2023-11-24T07:58:22.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.838+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "746138ca-9c6f-4895-a296-bd6778103fed", "id": 5} v 0) v1 -- 0x7f9cd0072a30 con 0x7f9cd006c710 2023-11-24T07:58:22.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.849+0000 7f9cc6ffd700 1 -- 172.21.15.123:0/3052466357 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "746138ca-9c6f-4895-a296-bd6778103fed", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ccc017070 con 0x7f9cd006c710 2023-11-24T07:58:22.851 INFO:teuthology.orchestra.run.smithi123.stdout:5 2023-11-24T07:58:22.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 msgr2=0x7f9cd01428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd01428c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e870 tx=0x7f9ccc00eb80 comp rx=0 tx=0).stop 2023-11-24T07:58:22.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 shutdown_connections 2023-11-24T07:58:22.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cd0158310 0x7f9cd01e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd006c710 0x7f9cd01428c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 --2- 172.21.15.123:0/3052466357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd00addb0 0x7f9cd00ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 >> 172.21.15.123:0/3052466357 conn(0x7f9cd0071870 msgr2=0x7f9cd00b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 shutdown_connections 2023-11-24T07:58:22.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:22.850+0000 7f9cd739e700 1 -- 172.21.15.123:0/3052466357 wait complete. 2023-11-24T07:58:22.863 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 86f44387-1510-4912-b4bc-3ee6550bf505 6 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.007+0000 7fceddc16700 1 Processor -- start 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.007+0000 7fceddc16700 1 -- start start 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.008+0000 7fceddc16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e380 0x7fced8165000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.008+0000 7fceddc16700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced806d290 con 0x7fced815e960 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.008+0000 7fceddc16700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced806e100 con 0x7fced81405e0 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.008+0000 7fceddc16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced81409c0 con 0x7fced815e380 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.008+0000 7fced6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fced815e960 0x7fced815dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.123:44278/0 (socket says 172.21.15.123:44278) 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.008+0000 7fced6ffd700 1 -- 172.21.15.123:0/1605734085 learned_addr learned my addr 172.21.15.123:0/1605734085 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-11-24T07:58:23.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced7fff700 1 --2- 172.21.15.123:0/1605734085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e380 0x7fced8165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177271678 0 0) 0x7fced806d290 con 0x7fced815e960 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec00035b0 con 0x7fced815e960 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772521343 0 0) 0x7fced806e100 con 0x7fced81405e0 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced806d290 con 0x7fced81405e0 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced7fff700 1 -- 172.21.15.123:0/1605734085 >> v1:172.21.15.123:6789/0 conn(0x7fced81405e0 legacy=0x7fced806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced7fff700 1 -- 172.21.15.123:0/1605734085 >> v1:172.21.15.32:6789/0 conn(0x7fced815e960 legacy=0x7fced815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced7fff700 1 -- 172.21.15.123:0/1605734085 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced806d4f0 con 0x7fced815e380 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced7fff700 1 --2- 172.21.15.123:0/1605734085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e380 0x7fced8165000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcec800b180 tx=0x7fcec800b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a574950c7c107ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec800e070 con 0x7fced815e380 2023-11-24T07:58:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec800bda0 con 0x7fced815e380 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.009+0000 7fced67fc700 1 -- 172.21.15.123:0/1605734085 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8007b10 con 0x7fced815e380 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 -- 172.21.15.123:0/1605734085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e380 msgr2=0x7fced8165000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 --2- 172.21.15.123:0/1605734085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e380 0x7fced8165000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcec800b180 tx=0x7fcec800b490 comp rx=0 tx=0).stop 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 -- 172.21.15.123:0/1605734085 shutdown_connections 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 --2- 172.21.15.123:0/1605734085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e380 0x7fced8165000 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 -- 172.21.15.123:0/1605734085 >> 172.21.15.123:0/1605734085 conn(0x7fced806b930 msgr2=0x7fced80b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 -- 172.21.15.123:0/1605734085 shutdown_connections 2023-11-24T07:58:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.010+0000 7fceddc16700 1 -- 172.21.15.123:0/1605734085 wait complete. 2023-11-24T07:58:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.011+0000 7fceddc16700 1 Processor -- start 2023-11-24T07:58:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.011+0000 7fceddc16700 1 -- start start 2023-11-24T07:58:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fceddc16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fced81405e0 0x7fced8152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fceddc16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced815e380 0x7fced813ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fceddc16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e960 0x7fced813e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fceddc16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced80abc60 con 0x7fced81405e0 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fceddc16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced815b770 con 0x7fced815e380 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fceddc16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced815b490 con 0x7fced815e960 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fced6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced815e380 0x7fced813ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fced6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced815e380 0x7fced813ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56724/0 (socket says 172.21.15.123:56724) 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fced6ffd700 1 -- 172.21.15.123:0/2458356024 learned_addr learned my addr 172.21.15.123:0/2458356024 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fced7fff700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e960 0x7fced813e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.012+0000 7fced77fe700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fced81405e0 0x7fced8152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fced7fff700 1 -- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced815e380 msgr2=0x7fced813ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fced7fff700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced815e380 0x7fced813ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fced7fff700 1 -- 172.21.15.123:0/2458356024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fced81405e0 msgr2=0x7fced8152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fced7fff700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fced81405e0 0x7fced8152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fced7fff700 1 -- 172.21.15.123:0/2458356024 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8009dc0 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fced7fff700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e960 0x7fced813e2f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fced8143e70 tx=0x7fcec8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec801c650 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec801c7b0 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced80ab5a0 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec801cb60 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced8150420 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcec8025410 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.013+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fcec80129e0 con 0x7fced815e960 2023-11-24T07:58:23.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.014+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced8068480 con 0x7fced815e960 2023-11-24T07:58:23.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.016+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fcec8020030 con 0x7fced815e960 2023-11-24T07:58:23.101 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.099+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "86f44387-1510-4912-b4bc-3ee6550bf505", "id": 6} v 0) v1 -- 0x7fced8068fb0 con 0x7fced815e960 2023-11-24T07:58:23.109 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.107+0000 7fcebffff700 1 -- 172.21.15.123:0/2458356024 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "86f44387-1510-4912-b4bc-3ee6550bf505", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcec80330b0 con 0x7fced815e960 2023-11-24T07:58:23.109 INFO:teuthology.orchestra.run.smithi123.stdout:6 2023-11-24T07:58:23.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e960 msgr2=0x7fced813e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e960 0x7fced813e2f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fced8143e70 tx=0x7fcec8004750 comp rx=0 tx=0).stop 2023-11-24T07:58:23.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 shutdown_connections 2023-11-24T07:58:23.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fced815e960 0x7fced813e2f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fced81405e0 0x7fced8152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 --2- 172.21.15.123:0/2458356024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced815e380 0x7fced813ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 >> 172.21.15.123:0/2458356024 conn(0x7fced806b930 msgr2=0x7fced8140b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 shutdown_connections 2023-11-24T07:58:23.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.108+0000 7fceddc16700 1 -- 172.21.15.123:0/2458356024 wait complete. 2023-11-24T07:58:23.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new c822d5cc-2051-4923-8ddb-32ead3b8cf10 7 2023-11-24T07:58:23.208 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:23.207+0000 7f82285ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T07:58:23.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 Processor -- start 2023-11-24T07:58:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 -- start start 2023-11-24T07:58:23.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38e415b370 0x7f38e415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e40b3c60 0x7f38e40b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40ae5e0 con 0x7f38e41587f0 2023-11-24T07:58:23.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40b4630 con 0x7f38e415b370 2023-11-24T07:58:23.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.243+0000 7f38e9d81700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e415be30 con 0x7f38e40b3c60 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38e415b370 0x7f38e415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f38e41587f0 0x7f38e406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47008/0 (socket says 172.21.15.123:47008) 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e37fe700 1 -- 172.21.15.123:0/2532992065 learned_addr learned my addr 172.21.15.123:0/2532992065 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e40b3c60 0x7f38e40b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e3fff700 1 -- 172.21.15.123:0/2532992065 >> v2:172.21.15.123:3300/0 conn(0x7f38e415b370 msgr2=0x7f38e415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e3fff700 1 --2- 172.21.15.123:0/2532992065 >> v2:172.21.15.123:3300/0 conn(0x7f38e415b370 0x7f38e415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e3fff700 1 -- 172.21.15.123:0/2532992065 >> v1:172.21.15.123:6789/0 conn(0x7f38e41587f0 legacy=0x7f38e406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e3fff700 1 -- 172.21.15.123:0/2532992065 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e415c110 con 0x7f38e40b3c60 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e27fc700 1 -- 172.21.15.123:0/2532992065 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257217130 0 0) 0x7f38e40ae5e0 con 0x7f38e41587f0 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e2ffd700 1 --2- 172.21.15.123:0/2532992065 >> v2:172.21.15.123:3300/0 conn(0x7f38e415b370 0x7f38e415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:23.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e3fff700 1 --2- 172.21.15.123:0/2532992065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e40b3c60 0x7f38e40b40f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009dc0 tx=0x7f38dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bcc057fefe520a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e27fc700 1 -- 172.21.15.123:0/2532992065 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00e070 con 0x7f38e40b3c60 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e27fc700 1 -- 172.21.15.123:0/2532992065 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00bae0 con 0x7f38e40b3c60 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.244+0000 7f38e27fc700 1 -- 172.21.15.123:0/2532992065 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00be90 con 0x7f38e40b3c60 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 -- 172.21.15.123:0/2532992065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e40b3c60 msgr2=0x7f38e40b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 --2- 172.21.15.123:0/2532992065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e40b3c60 0x7f38e40b40f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009dc0 tx=0x7f38dc00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 -- 172.21.15.123:0/2532992065 shutdown_connections 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 --2- 172.21.15.123:0/2532992065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e40b3c60 0x7f38e40b40f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 --2- 172.21.15.123:0/2532992065 >> v2:172.21.15.123:3300/0 conn(0x7f38e415b370 0x7f38e415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 -- 172.21.15.123:0/2532992065 >> 172.21.15.123:0/2532992065 conn(0x7f38e40d0bf0 msgr2=0x7f38e40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 -- 172.21.15.123:0/2532992065 shutdown_connections 2023-11-24T07:58:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.245+0000 7f38e9d81700 1 -- 172.21.15.123:0/2532992065 wait complete. 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.246+0000 7f38e9d81700 1 Processor -- start 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.247+0000 7f38e9d81700 1 -- start start 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.247+0000 7f38e9d81700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e40b3c60 0x7f38e4155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.247+0000 7f38e9d81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e41587f0 0x7f38e40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e9d81700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 0x7f38e40c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e9d81700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41f2280 con 0x7f38e41587f0 2023-11-24T07:58:23.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e9d81700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41562e0 con 0x7f38e40b3c60 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e9d81700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41f1340 con 0x7f38e415b370 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 0x7f38e40c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 0x7f38e40c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:51230/0 (socket says 172.21.15.123:51230) 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 -- 172.21.15.123:0/1136713346 learned_addr learned my addr 172.21.15.123:0/1136713346 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e2ffd700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e41587f0 0x7f38e40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 -- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e40b3c60 msgr2=0x7f38e4155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e40b3c60 0x7f38e4155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 -- 172.21.15.123:0/1136713346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e41587f0 msgr2=0x7f38e40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e41587f0 0x7f38e40c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 -- 172.21.15.123:0/1136713346 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80094a0 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e3fff700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 0x7f38e40c6110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f38dc014040 tx=0x7f38dc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0045d0 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc020070 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc009a70 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc004970 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.248+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41f18e0 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.249+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f38dc027780 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.249+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f38dc022070 con 0x7f38e415b370 2023-11-24T07:58:23.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.249+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e4068fb0 con 0x7f38e415b370 2023-11-24T07:58:23.255 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:23.252+0000 7f00d0609200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T07:58:23.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.251+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f38dc02c060 con 0x7f38e415b370 2023-11-24T07:58:23.280 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:23.279+0000 7f82285ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T07:58:23.326 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:23.324+0000 7f00d0609200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T07:58:23.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.335+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c822d5cc-2051-4923-8ddb-32ead3b8cf10", "id": 7} v 0) v1 -- 0x7f38e40d0260 con 0x7f38e415b370 2023-11-24T07:58:23.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.339+0000 7f38cbfff700 1 -- 172.21.15.123:0/1136713346 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c822d5cc-2051-4923-8ddb-32ead3b8cf10", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f38dc0128e0 con 0x7f38e415b370 2023-11-24T07:58:23.340 INFO:teuthology.orchestra.run.smithi123.stdout:7 2023-11-24T07:58:23.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.339+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 msgr2=0x7f38e40c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.339+0000 7f38e9d81700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 0x7f38e40c6110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f38dc014040 tx=0x7f38dc004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 shutdown_connections 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38e415b370 0x7f38e40c6110 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e40b3c60 0x7f38e4155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 --2- 172.21.15.123:0/1136713346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e41587f0 0x7f38e40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 >> 172.21.15.123:0/1136713346 conn(0x7f38e40d0bf0 msgr2=0x7f38e406f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 shutdown_connections 2023-11-24T07:58:23.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-11-24T07:58:23.340+0000 7f38e9d81700 1 -- 172.21.15.123:0/1136713346 wait complete. 2023-11-24T07:58:23.353 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T07:58:23.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T07:58:23.355 INFO:tasks.ceph.osd.0:Started 2023-11-24T07:58:23.355 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T07:58:23.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T07:58:23.357 INFO:tasks.ceph.osd.1:Started 2023-11-24T07:58:23.358 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T07:58:23.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T07:58:23.360 INFO:tasks.ceph.osd.2:Started 2023-11-24T07:58:23.360 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T07:58:23.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T07:58:23.362 INFO:tasks.ceph.osd.3:Started 2023-11-24T07:58:23.362 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T07:58:23.362 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T07:58:23.364 INFO:tasks.ceph.osd.4:Started 2023-11-24T07:58:23.364 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T07:58:23.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T07:58:23.366 INFO:tasks.ceph.osd.5:Started 2023-11-24T07:58:23.366 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T07:58:23.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T07:58:23.368 INFO:tasks.ceph.osd.6:Started 2023-11-24T07:58:23.368 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T07:58:23.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T07:58:23.370 INFO:tasks.ceph.osd.7:Started 2023-11-24T07:58:23.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:23.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.528+0000 7fb9517c4700 1 Processor -- start 2023-11-24T07:58:23.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.528+0000 7fb9517c4700 1 -- start start 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb9517c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb94c0710d0 0x7fb94c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb9517c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c0cf220 con 0x7fb94c15ad40 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb9517c4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c0afb50 con 0x7fb94c072ff0 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb9517c4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0b5b10 con 0x7fb94c0710d0 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb94affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb94c15ad40 0x7fb94c0728e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57612/0 (socket says 172.21.15.32:57612) 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb94affd700 1 -- 172.21.15.32:0/3375630579 learned_addr learned my addr 172.21.15.32:0/3375630579 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb94b7fe700 1 --2- 172.21.15.32:0/3375630579 >> v2:172.21.15.123:3300/0 conn(0x7fb94c0710d0 0x7fb94c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110341448 0 0) 0x7fb94c0afb50 con 0x7fb94c072ff0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9340035b0 con 0x7fb94c072ff0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339999739 0 0) 0x7fb94c0cf220 con 0x7fb94c15ad40 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb94c0afb50 con 0x7fb94c15ad40 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb94b7fe700 1 -- 172.21.15.32:0/3375630579 >> v1:172.21.15.123:6789/0 conn(0x7fb94c072ff0 legacy=0x7fb94c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb94b7fe700 1 -- 172.21.15.32:0/3375630579 >> v1:172.21.15.32:6789/0 conn(0x7fb94c15ad40 legacy=0x7fb94c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.529+0000 7fb94b7fe700 1 -- 172.21.15.32:0/3375630579 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0b5df0 con 0x7fb94c0710d0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb94b7fe700 1 --2- 172.21.15.32:0/3375630579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c0710d0 0x7fb94c0cf6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb944009d90 tx=0x7fb94400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3115e54064a4fbcb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400e070 con 0x7fb94c0710d0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94400bb20 con 0x7fb94c0710d0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb949ffb700 1 -- 172.21.15.32:0/3375630579 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944012430 con 0x7fb94c0710d0 2023-11-24T07:58:23.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 -- 172.21.15.32:0/3375630579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c0710d0 msgr2=0x7fb94c0cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 --2- 172.21.15.32:0/3375630579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c0710d0 0x7fb94c0cf6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb944009d90 tx=0x7fb94400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 -- 172.21.15.32:0/3375630579 shutdown_connections 2023-11-24T07:58:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 --2- 172.21.15.32:0/3375630579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c0710d0 0x7fb94c0cf6b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 -- 172.21.15.32:0/3375630579 >> 172.21.15.32:0/3375630579 conn(0x7fb94c06cec0 msgr2=0x7fb94c06d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 -- 172.21.15.32:0/3375630579 shutdown_connections 2023-11-24T07:58:23.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.530+0000 7fb9517c4700 1 -- 172.21.15.32:0/3375630579 wait complete. 2023-11-24T07:58:23.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 Processor -- start 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 -- start start 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 0x7fb94c1f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb94c072ff0 0x7fb94c14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c15ad40 0x7fb94c1f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1f3450 con 0x7fb94c0710d0 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1503a0 con 0x7fb94c15ad40 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb9517c4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c14d890 con 0x7fb94c072ff0 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.532+0000 7fb94affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 0x7fb94c1f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 0x7fb94c1f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55906/0 (socket says 172.21.15.32:55906) 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 -- 172.21.15.32:0/2375895780 learned_addr learned my addr 172.21.15.32:0/2375895780 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94b7fe700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c15ad40 0x7fb94c1f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94a7fc700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb94c072ff0 0x7fb94c14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 -- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb94c072ff0 msgr2=0x7fb94c14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb94c072ff0 0x7fb94c14cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 -- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c15ad40 msgr2=0x7fb94c1f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c15ad40 0x7fb94c1f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 -- 172.21.15.32:0/2375895780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944009a70 con 0x7fb94c0710d0 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb94affd700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 0x7fb94c1f28d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb93c001eb0 tx=0x7fb93c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c005d70 con 0x7fb94c0710d0 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c0070d0 con 0x7fb94c0710d0 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c14db70 con 0x7fb94c0710d0 2023-11-24T07:58:23.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c007460 con 0x7fb94c0710d0 2023-11-24T07:58:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.533+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c14e130 con 0x7fb94c0710d0 2023-11-24T07:58:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.534+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb93c01c440 con 0x7fb94c0710d0 2023-11-24T07:58:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.534+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb93c012ad0 con 0x7fb94c0710d0 2023-11-24T07:58:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.534+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c14e4a0 con 0x7fb94c0710d0 2023-11-24T07:58:23.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.536+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb93c017070 con 0x7fb94c0710d0 2023-11-24T07:58:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.608+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb94c14e2c0 con 0x7fb94c0710d0 2023-11-24T07:58:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.608+0000 7fb9337fe700 1 -- 172.21.15.32:0/2375895780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb93c012400 con 0x7fb94c0710d0 2023-11-24T07:58:23.610 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:23.610 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:23.336987+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 msgr2=0x7fb94c1f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 0x7fb94c1f28d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb93c001eb0 tx=0x7fb93c00c7b0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 shutdown_connections 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c15ad40 0x7fb94c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb94c0710d0 0x7fb94c1f28d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 --2- 172.21.15.32:0/2375895780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb94c072ff0 0x7fb94c14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.609+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 >> 172.21.15.32:0/2375895780 conn(0x7fb94c06cec0 msgr2=0x7fb94c15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.610+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 shutdown_connections 2023-11-24T07:58:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:23.610+0000 7fb9517c4700 1 -- 172.21.15.32:0/2375895780 wait complete. 2023-11-24T07:58:23.623 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T07:58:23.624 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T07:58:23.965 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-11-24T07:58:23.964+0000 7f786d141640 -1 Falling back to public interface 2023-11-24T07:58:23.974 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-11-24T07:58:23.972+0000 7f3ca0dcb640 -1 Falling back to public interface 2023-11-24T07:58:23.978 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-11-24T07:58:23.976+0000 7f64bd446640 -1 Falling back to public interface 2023-11-24T07:58:23.979 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-11-24T07:58:23.977+0000 7f6c7160a640 -1 Falling back to public interface 2023-11-24T07:58:23.989 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-11-24T07:58:23.987+0000 7f7b54753640 -1 Falling back to public interface 2023-11-24T07:58:23.990 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-11-24T07:58:23.988+0000 7fc37f696640 -1 Falling back to public interface 2023-11-24T07:58:23.991 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-11-24T07:58:23.989+0000 7f4463c2c640 -1 Falling back to public interface 2023-11-24T07:58:24.000 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-11-24T07:58:23.998+0000 7f4816c16640 -1 Falling back to public interface 2023-11-24T07:58:24.017 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:24.016+0000 7f82285ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:58:24.093 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:24.091+0000 7f82285ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T07:58:24.102 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:24.101+0000 7f00d0609200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:58:24.183 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:24.182+0000 7f00d0609200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T07:58:24.232 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:24.231+0000 7f82285ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:58:24.331 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:24.330+0000 7f00d0609200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:58:24.430 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:24.467 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:24.466+0000 7f82285ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T07:58:24.546 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.545+0000 7ff59b3da700 1 Processor -- start 2023-11-24T07:58:24.547 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.545+0000 7ff59b3da700 1 -- start start 2023-11-24T07:58:24.549 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff59b3da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5940d02c0 0x7ff5940d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff59b3da700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5940d0740 0x7ff5940d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff59b3da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940d0c40 con 0x7ff59415bf60 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff59b3da700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59415c340 con 0x7ff5940d02c0 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff59b3da700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59406f220 con 0x7ff5940d0740 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff598975700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5940d02c0 0x7ff5940d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff598975700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5940d02c0 0x7ff5940d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55914/0 (socket says 172.21.15.32:55914) 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff598975700 1 -- 172.21.15.32:0/2761471113 learned_addr learned my addr 172.21.15.32:0/2761471113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.546+0000 7ff599176700 1 --1- 172.21.15.32:0/2761471113 >> v1:172.21.15.32:6789/0 conn(0x7ff59415bf60 0x7ff5940cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57624/0 (socket says 172.21.15.32:57624) 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff599977700 1 --2- 172.21.15.32:0/2761471113 >> v2:172.21.15.123:3300/0 conn(0x7ff5940d0740 0x7ff5940d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.550 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff598975700 1 -- 172.21.15.32:0/2761471113 >> v2:172.21.15.123:3300/0 conn(0x7ff5940d0740 msgr2=0x7ff5940d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff598975700 1 --2- 172.21.15.32:0/2761471113 >> v2:172.21.15.123:3300/0 conn(0x7ff5940d0740 0x7ff5940d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff598975700 1 -- 172.21.15.32:0/2761471113 >> v1:172.21.15.32:6789/0 conn(0x7ff59415bf60 legacy=0x7ff5940cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff598975700 1 -- 172.21.15.32:0/2761471113 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940d25d0 con 0x7ff5940d02c0 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff58bfff700 1 -- 172.21.15.32:0/2761471113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22243593 0 0) 0x7ff5940d0c40 con 0x7ff59415bf60 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff598975700 1 --2- 172.21.15.32:0/2761471113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d02c0 0x7ff5940d1b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff59406efc0 tx=0x7ff584009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20cd35947e0115 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff58bfff700 1 -- 172.21.15.32:0/2761471113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584011830 con 0x7ff5940d02c0 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff58bfff700 1 -- 172.21.15.32:0/2761471113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff584011990 con 0x7ff5940d02c0 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.547+0000 7ff58bfff700 1 -- 172.21.15.32:0/2761471113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584011d20 con 0x7ff5940d02c0 2023-11-24T07:58:24.551 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 -- 172.21.15.32:0/2761471113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d02c0 msgr2=0x7ff5940d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 --2- 172.21.15.32:0/2761471113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d02c0 0x7ff5940d1b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff59406efc0 tx=0x7ff584009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 -- 172.21.15.32:0/2761471113 shutdown_connections 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 --2- 172.21.15.32:0/2761471113 >> v2:172.21.15.123:3300/0 conn(0x7ff5940d0740 0x7ff5940d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 --2- 172.21.15.32:0/2761471113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d02c0 0x7ff5940d1b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 -- 172.21.15.32:0/2761471113 >> 172.21.15.32:0/2761471113 conn(0x7ff59409aa00 msgr2=0x7ff59409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 -- 172.21.15.32:0/2761471113 shutdown_connections 2023-11-24T07:58:24.552 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.548+0000 7ff59b3da700 1 -- 172.21.15.32:0/2761471113 wait complete. 2023-11-24T07:58:24.554 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.550+0000 7ff59b3da700 1 Processor -- start 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.550+0000 7ff59b3da700 1 -- start start 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff59b3da700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5940d02c0 0x7ff59415a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff59b3da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 0x7ff59415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff59b3da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59415bf60 0x7ff59415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff59b3da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940c6dc0 con 0x7ff5940d0740 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff59b3da700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59406c950 con 0x7ff59415bf60 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff59b3da700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940c6420 con 0x7ff5940d02c0 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff598975700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 0x7ff59415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff598975700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 0x7ff59415ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55920/0 (socket says 172.21.15.32:55920) 2023-11-24T07:58:24.555 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff598975700 1 -- 172.21.15.32:0/1581755243 learned_addr learned my addr 172.21.15.32:0/1581755243 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff599977700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59415bf60 0x7ff59415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.551+0000 7ff599176700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5940d02c0 0x7ff59415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff598975700 1 -- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5940d02c0 msgr2=0x7ff59415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff598975700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5940d02c0 0x7ff59415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff598975700 1 -- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59415bf60 msgr2=0x7ff59415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff598975700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59415bf60 0x7ff59415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff598975700 1 -- 172.21.15.32:0/1581755243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584009550 con 0x7ff5940d0740 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff598975700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 0x7ff59415ee30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5940715e0 tx=0x7ff584004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584004510 con 0x7ff5940d0740 2023-11-24T07:58:24.556 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5840036a0 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5940c6700 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584003a10 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.552+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59415aa60 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.553+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff58401a430 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.553+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff584021030 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.553+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5940d0130 con 0x7ff5940d0740 2023-11-24T07:58:24.557 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.555+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff58402b060 con 0x7ff5940d0740 2023-11-24T07:58:24.583 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:24.582+0000 7f00d0609200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T07:58:24.629 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.627+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff594068fb0 con 0x7ff5940d0740 2023-11-24T07:58:24.630 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.628+0000 7ff589ffb700 1 -- 172.21.15.32:0/1581755243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff584015020 con 0x7ff5940d0740 2023-11-24T07:58:24.631 INFO:teuthology.misc.health.smithi032.stdout: 2023-11-24T07:58:24.631 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:23.336987+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:24.632 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 msgr2=0x7ff59415ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.632 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 0x7ff59415ee30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5940715e0 tx=0x7ff584004750 comp rx=0 tx=0).stop 2023-11-24T07:58:24.632 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 shutdown_connections 2023-11-24T07:58:24.632 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59415bf60 0x7ff59415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.632 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff5940d02c0 0x7ff59415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.632 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 --2- 172.21.15.32:0/1581755243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d0740 0x7ff59415ee30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.633 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 >> 172.21.15.32:0/1581755243 conn(0x7ff59409aa00 msgr2=0x7ff5940b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.633 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 shutdown_connections 2023-11-24T07:58:24.633 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:24.629+0000 7ff59b3da700 1 -- 172.21.15.32:0/1581755243 wait complete. 2023-11-24T07:58:24.642 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T07:58:24.679 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:24.677+0000 7f82285ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T07:58:24.803 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:24.802+0000 7f00d0609200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T07:58:24.813 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:24.811+0000 7f82285ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T07:58:24.943 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:24.942+0000 7f00d0609200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T07:58:25.032 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:25.030+0000 7f82285ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T07:58:25.172 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:25.171+0000 7f00d0609200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T07:58:25.187 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:25.186+0000 7f82285ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T07:58:25.300 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:25.298+0000 7f82285ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T07:58:25.334 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:25.332+0000 7f00d0609200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T07:58:25.455 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:25.453+0000 7f00d0609200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T07:58:25.622 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:25.620+0000 7f82285ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T07:58:25.722 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:25.720+0000 7f82285ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T07:58:25.790 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:25.789+0000 7f00d0609200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T07:58:25.894 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:25.893+0000 7f00d0609200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T07:58:25.910 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:25.908+0000 7f82285ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T07:58:26.091 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:26.089+0000 7f00d0609200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T07:58:26.508 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:26.506+0000 7f82285ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T07:58:26.735 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-11-24T07:58:26.733+0000 7f00d0609200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T07:58:27.475 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-11-24T07:58:27.473+0000 7f786d141640 -1 osd.1 0 log_to_monitors true 2023-11-24T07:58:27.504 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-11-24T07:58:27.503+0000 7f6c7160a640 -1 osd.3 0 log_to_monitors true 2023-11-24T07:58:27.521 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-11-24T07:58:27.519+0000 7f64bd446640 -1 osd.4 0 log_to_monitors true 2023-11-24T07:58:27.522 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-11-24T07:58:27.513+0000 7fc37f696640 -1 osd.5 0 log_to_monitors true 2023-11-24T07:58:27.533 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:27.531+0000 7f81e0541700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T07:58:27.533 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-11-24T07:58:27.532+0000 7f81e0541700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T07:58:27.540 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-11-24T07:58:27.539+0000 7f3ca0dcb640 -1 osd.6 0 log_to_monitors true 2023-11-24T07:58:27.543 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-11-24T07:58:27.541+0000 7f7b54753640 -1 osd.7 0 log_to_monitors true 2023-11-24T07:58:27.559 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-11-24T07:58:27.558+0000 7f4816c16640 -1 osd.2 0 log_to_monitors true 2023-11-24T07:58:27.565 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-11-24T07:58:27.564+0000 7f4463c2c640 -1 osd.0 0 log_to_monitors true 2023-11-24T07:58:29.946 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-11-24T07:58:29.944+0000 7f480a7cd700 -1 osd.2 0 waiting for initial osdmap 2023-11-24T07:58:31.883 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-11-24T07:58:30.599+0000 7fc37324d700 -1 osd.5 0 waiting for initial osdmap 2023-11-24T07:58:31.884 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-11-24T07:58:29.958+0000 7f4802bad700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.884 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-11-24T07:58:30.600+0000 7f7b4830a700 -1 osd.7 0 waiting for initial osdmap 2023-11-24T07:58:31.884 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-11-24T07:58:29.972+0000 7f6c651c1700 -1 osd.3 0 waiting for initial osdmap 2023-11-24T07:58:31.885 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-11-24T07:58:29.973+0000 7f7860cf8700 -1 osd.1 0 waiting for initial osdmap 2023-11-24T07:58:31.885 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-11-24T07:58:29.981+0000 7f44577e3700 -1 osd.0 0 waiting for initial osdmap 2023-11-24T07:58:31.885 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-11-24T07:58:30.605+0000 7fc36b62d700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.885 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-11-24T07:58:29.984+0000 7f6c5d5a1700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.886 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-11-24T07:58:30.608+0000 7f64b0ffd700 -1 osd.4 0 waiting for initial osdmap 2023-11-24T07:58:31.886 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-11-24T07:58:29.988+0000 7f78590d8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.886 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-11-24T07:58:30.615+0000 7f7b406ea700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.886 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-11-24T07:58:29.994+0000 7f444fbc3700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.887 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-11-24T07:58:30.625+0000 7f64a93dd700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:31.887 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-11-24T07:58:30.667+0000 7f3c94982700 -1 osd.6 0 waiting for initial osdmap 2023-11-24T07:58:31.887 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-11-24T07:58:30.680+0000 7f3c8cd62700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:58:32.689 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:32.817 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.815+0000 7f5b8310b700 1 Processor -- start 2023-11-24T07:58:32.817 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.815+0000 7f5b8310b700 1 -- start start 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b8310b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b7c0aff30 0x7f5b7c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b8310b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c15ec90 con 0x7f5b7c0b0390 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b8310b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c15ef70 con 0x7f5b7c158910 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b8310b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c158cf0 con 0x7f5b7c0aff30 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b7c0aff30 0x7f5b7c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b80ea7700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5b7c158910 0x7f5b7c0b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:44622/0 (socket says 172.21.15.32:44622) 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b816a8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b7c0b0390 0x7f5b7c162210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57872/0 (socket says 172.21.15.32:57872) 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b7c0aff30 0x7f5b7c06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56158/0 (socket says 172.21.15.32:56158) 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.816+0000 7f5b80ea7700 1 -- 172.21.15.32:0/3672530176 learned_addr learned my addr 172.21.15.32:0/3672530176 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7bfff700 1 -- 172.21.15.32:0/3672530176 >> v1:172.21.15.123:6789/0 conn(0x7f5b7c158910 legacy=0x7f5b7c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.819 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7bfff700 1 -- 172.21.15.32:0/3672530176 >> v1:172.21.15.32:6789/0 conn(0x7f5b7c0b0390 legacy=0x7f5b7c162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.820 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7bfff700 1 -- 172.21.15.32:0/3672530176 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c15f250 con 0x7f5b7c0aff30 2023-11-24T07:58:32.820 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7b7fe700 1 -- 172.21.15.32:0/3672530176 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504671271 0 0) 0x7f5b7c15ec90 con 0x7f5b7c0b0390 2023-11-24T07:58:32.820 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7bfff700 1 --2- 172.21.15.32:0/3672530176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0aff30 0x7f5b7c06c000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009840 tx=0x7f5b68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f1f565d0847dc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.820 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7b7fe700 1 -- 172.21.15.32:0/3672530176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68011830 con 0x7f5b7c0aff30 2023-11-24T07:58:32.820 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7b7fe700 1 -- 172.21.15.32:0/3672530176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68011990 con 0x7f5b7c0aff30 2023-11-24T07:58:32.820 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.817+0000 7f5b7b7fe700 1 -- 172.21.15.32:0/3672530176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68011d20 con 0x7f5b7c0aff30 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 -- 172.21.15.32:0/3672530176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0aff30 msgr2=0x7f5b7c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 --2- 172.21.15.32:0/3672530176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0aff30 0x7f5b7c06c000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009840 tx=0x7f5b68009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 -- 172.21.15.32:0/3672530176 shutdown_connections 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 --2- 172.21.15.32:0/3672530176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0aff30 0x7f5b7c06c000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 -- 172.21.15.32:0/3672530176 >> 172.21.15.32:0/3672530176 conn(0x7f5b7c071ae0 msgr2=0x7f5b7c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 -- 172.21.15.32:0/3672530176 shutdown_connections 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.818+0000 7f5b8310b700 1 -- 172.21.15.32:0/3672530176 wait complete. 2023-11-24T07:58:32.823 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.819+0000 7f5b8310b700 1 Processor -- start 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.819+0000 7f5b8310b700 1 -- start start 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b8310b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7c0aff30 0x7f5b7c14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b8310b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 0x7f5b7c152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b8310b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7c158910 0x7f5b7c160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b8310b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c14ec70 con 0x7f5b7c0b0390 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b8310b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c153b30 con 0x7f5b7c158910 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b8310b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c152eb0 con 0x7f5b7c0aff30 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 0x7f5b7c152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 0x7f5b7c152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56164/0 (socket says 172.21.15.32:56164) 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 -- 172.21.15.32:0/1327627719 learned_addr learned my addr 172.21.15.32:0/1327627719 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:32.824 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b816a8700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7c158910 0x7f5b7c160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b80ea7700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7c0aff30 0x7f5b7c14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 -- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7c0aff30 msgr2=0x7f5b7c14e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7c0aff30 0x7f5b7c14e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 -- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7c158910 msgr2=0x7f5b7c160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7c158910 0x7f5b7c160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b7bfff700 1 -- 172.21.15.32:0/1327627719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680094a0 con 0x7f5b7c0b0390 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.820+0000 7f5b80ea7700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7c0aff30 0x7f5b7c14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.821+0000 7f5b7bfff700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 0x7f5b7c152460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b680090b0 tx=0x7f5b68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.821+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6801a850 con 0x7f5b7c0b0390 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.821+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6801a9b0 con 0x7f5b7c0b0390 2023-11-24T07:58:32.825 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.821+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c153190 con 0x7f5b7c0b0390 2023-11-24T07:58:32.826 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.821+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6801ad20 con 0x7f5b7c0b0390 2023-11-24T07:58:32.826 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.821+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c153750 con 0x7f5b7c0b0390 2023-11-24T07:58:32.827 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.822+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c068480 con 0x7f5b7c0b0390 2023-11-24T07:58:32.827 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.822+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f5b68022070 con 0x7f5b7c0b0390 2023-11-24T07:58:32.827 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.823+0000 7f5b797fa700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5b6006ae40 0x7f5b6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.827 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.823+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5b6808a210 con 0x7f5b7c0b0390 2023-11-24T07:58:32.827 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.823+0000 7f5b80ea7700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5b6006ae40 0x7f5b6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.827 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.823+0000 7f5b80ea7700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5b6006ae40 0x7f5b6006d300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b70007fd0 tx=0x7f5b70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.829 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:32.828+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b68055910 con 0x7f5b7c0b0390 2023-11-24T07:58:33.031 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.030+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b7c068fb0 con 0x7f5b7c0b0390 2023-11-24T07:58:33.032 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.031+0000 7f5b797fa700 1 -- 172.21.15.32:0/1327627719 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12842 (secure 0 0 0) 0x7f5b68015090 con 0x7f5b7c0b0390 2023-11-24T07:58:33.032 INFO:teuthology.misc.health.smithi032.stdout: 2023-11-24T07:58:33.032 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":15,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:32.986828+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5b6006ae40 msgr2=0x7f5b6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5b6006ae40 0x7f5b6006d300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b70007fd0 tx=0x7f5b70009040 comp rx=0 tx=0).stop 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 msgr2=0x7f5b7c152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 0x7f5b7c152460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b680090b0 tx=0x7f5b68004750 comp rx=0 tx=0).stop 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 shutdown_connections 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5b6006ae40 0x7f5b6006d300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b7c158910 0x7f5b7c160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b7c0aff30 0x7f5b7c14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.035 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 --2- 172.21.15.32:0/1327627719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b7c0b0390 0x7f5b7c152460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.036 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 >> 172.21.15.32:0/1327627719 conn(0x7f5b7c071ae0 msgr2=0x7f5b7c071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.036 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 shutdown_connections 2023-11-24T07:58:33.036 INFO:teuthology.misc.health.smithi032.stderr:2023-11-24T07:58:33.033+0000 7f5b8310b700 1 -- 172.21.15.32:0/1327627719 wait complete. 2023-11-24T07:58:33.046 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T07:58:33.046 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T07:58:33.046 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:58:33.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:58:33.049 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:58:33.049 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:58:33.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:58:33.051 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:58:33.051 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:58:33.051 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:58:33.055 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:58:33.055 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:58:33.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:58:33.058 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:58:33.058 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T07:58:33.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:33.130 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-11-24T07:58:33.131 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-11-24T07:58:33.131 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-11-24T07:58:33.130+0000 7f7766ff9700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6833/0: (98) Address already in use 2023-11-24T07:58:33.151 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-11-24T07:58:33.152 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-11-24T07:58:33.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.188+0000 7f36df425700 1 Processor -- start 2023-11-24T07:58:33.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.189+0000 7f36df425700 1 -- start start 2023-11-24T07:58:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.189+0000 7f36df425700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f36d815e1e0 0x7f36d80d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.189+0000 7f36df425700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36d80d0f80 0x7f36d80b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.189+0000 7f36df425700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d80d20b0 con 0x7f36d80d3a00 2023-11-24T07:58:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36df425700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d80d3150 con 0x7f36d80d0f80 2023-11-24T07:58:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36df425700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d815e5c0 con 0x7f36d815e1e0 2023-11-24T07:58:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36d80d0f80 0x7f36d80b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36d80d0f80 0x7f36d80b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56200/0 (socket says 172.21.15.32:56200) 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 -- 172.21.15.32:0/2601934763 learned_addr learned my addr 172.21.15.32:0/2601934763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd1c1700 1 --2- 172.21.15.32:0/2601934763 >> v2:172.21.15.123:3300/0 conn(0x7f36d815e1e0 0x7f36d80d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 -- 172.21.15.32:0/2601934763 >> v2:172.21.15.123:3300/0 conn(0x7f36d815e1e0 msgr2=0x7f36d80d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 --2- 172.21.15.32:0/2601934763 >> v2:172.21.15.123:3300/0 conn(0x7f36d815e1e0 0x7f36d80d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 -- 172.21.15.32:0/2601934763 >> v1:172.21.15.123:6789/0 conn(0x7f36d80d3a00 legacy=0x7f36d80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 -- 172.21.15.32:0/2601934763 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80d2390 con 0x7f36d80d0f80 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36dd9c2700 1 --2- 172.21.15.32:0/2601934763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d80b6ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f36d80d0840 tx=0x7f36c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc00466efc46a2e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36d7fff700 1 -- 172.21.15.32:0/2601934763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c800e070 con 0x7f36d80d0f80 2023-11-24T07:58:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36d7fff700 1 -- 172.21.15.32:0/2601934763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c8003a20 con 0x7f36d80d0f80 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.190+0000 7f36d7fff700 1 -- 172.21.15.32:0/2601934763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8003dd0 con 0x7f36d80d0f80 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 -- 172.21.15.32:0/2601934763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 msgr2=0x7f36d80b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 --2- 172.21.15.32:0/2601934763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d80b6ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f36d80d0840 tx=0x7f36c800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 -- 172.21.15.32:0/2601934763 shutdown_connections 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 --2- 172.21.15.32:0/2601934763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d80b6ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 --2- 172.21.15.32:0/2601934763 >> v2:172.21.15.123:3300/0 conn(0x7f36d815e1e0 0x7f36d80d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 -- 172.21.15.32:0/2601934763 >> 172.21.15.32:0/2601934763 conn(0x7f36d80b5b50 msgr2=0x7f36d80b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 -- 172.21.15.32:0/2601934763 shutdown_connections 2023-11-24T07:58:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.191+0000 7f36df425700 1 -- 172.21.15.32:0/2601934763 wait complete. 2023-11-24T07:58:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.192+0000 7f36df425700 1 Processor -- start 2023-11-24T07:58:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.192+0000 7f36df425700 1 -- start start 2023-11-24T07:58:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36df425700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d8158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36df425700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d80d3a00 0x7f36d815f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36df425700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36d815e1e0 0x7f36d815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36df425700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d815e700 con 0x7f36d80d0f80 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36df425700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d815e950 con 0x7f36d80d3a00 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36df425700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d815ec30 con 0x7f36d815e1e0 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d8158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d8158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56202/0 (socket says 172.21.15.32:56202) 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 -- 172.21.15.32:0/1417889946 learned_addr learned my addr 172.21.15.32:0/1417889946 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dc9c0700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d80d3a00 0x7f36d815f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd9c2700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36d815e1e0 0x7f36d815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 -- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36d815e1e0 msgr2=0x7f36d815fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36d815e1e0 0x7f36d815fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 -- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d80d3a00 msgr2=0x7f36d815f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d80d3a00 0x7f36d815f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 -- 172.21.15.32:0/1417889946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c8009b30 con 0x7f36d80d0f80 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dc9c0700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d80d3a00 0x7f36d815f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd9c2700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36d815e1e0 0x7f36d815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36dd1c1700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d8158500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f36d000d800 tx=0x7f36d000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.193+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00125f0 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.194+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0012750 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.194+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0012b00 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.194+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d815ef10 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.194+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d8161170 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.195+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d815f560 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.195+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f36d0012c60 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.196+0000 7f36d5ffb700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f36c406ae40 0x7f36c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.196+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f36d00894d0 con 0x7f36d80d0f80 2023-11-24T07:58:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.196+0000 7f36dc9c0700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f36c406ae40 0x7f36c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.196+0000 7f36dc9c0700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f36c406ae40 0x7f36c406d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36d80d3e30 tx=0x7f36cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.200+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36d0054a50 con 0x7f36d80d0f80 2023-11-24T07:58:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.407+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36d81613f0 con 0x7f36d80d0f80 2023-11-24T07:58:33.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.408+0000 7f36d5ffb700 1 -- 172.21.15.32:0/1417889946 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12842 (secure 0 0 0) 0x7f36d0058070 con 0x7f36d80d0f80 2023-11-24T07:58:33.409 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:33.410 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:32.986828+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:33.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f36c406ae40 msgr2=0x7f36c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f36c406ae40 0x7f36c406d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36d80d3e30 tx=0x7f36cc009040 comp rx=0 tx=0).stop 2023-11-24T07:58:33.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 msgr2=0x7f36d8158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d8158500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f36d000d800 tx=0x7f36d000dbc0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 shutdown_connections 2023-11-24T07:58:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f36c406ae40 0x7f36c406d300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36d815e1e0 0x7f36d815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d80d0f80 0x7f36d8158500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 --2- 172.21.15.32:0/1417889946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d80d3a00 0x7f36d815f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 >> 172.21.15.32:0/1417889946 conn(0x7f36d80b5b50 msgr2=0x7f36d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 shutdown_connections 2023-11-24T07:58:33.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.410+0000 7f36df425700 1 -- 172.21.15.32:0/1417889946 wait complete. 2023-11-24T07:58:33.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:33.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.550+0000 7f5612105700 1 Processor -- start 2023-11-24T07:58:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.550+0000 7f5612105700 1 -- start start 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f5612105700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f560c15be70 0x7f560c0cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f5612105700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f560c0746a0 0x7f560c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f5612105700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c0b5ee0 con 0x7f560c0add40 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f5612105700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c06bfc0 con 0x7f560c15be70 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f5612105700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c06ba00 con 0x7f560c0746a0 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f560b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f560c15be70 0x7f560c0cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f560affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f560c0add40 0x7f560c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57924/0 (socket says 172.21.15.32:57924) 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.551+0000 7f560affd700 1 -- 172.21.15.32:0/3844904422 learned_addr learned my addr 172.21.15.32:0/3844904422 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560bfff700 1 --2- 172.21.15.32:0/3844904422 >> v2:172.21.15.123:3300/0 conn(0x7f560c0746a0 0x7f560c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560b7fe700 1 -- 172.21.15.32:0/3844904422 >> v2:172.21.15.123:3300/0 conn(0x7f560c0746a0 msgr2=0x7f560c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560b7fe700 1 --2- 172.21.15.32:0/3844904422 >> v2:172.21.15.123:3300/0 conn(0x7f560c0746a0 0x7f560c0b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560b7fe700 1 -- 172.21.15.32:0/3844904422 >> v1:172.21.15.32:6789/0 conn(0x7f560c0add40 legacy=0x7f560c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560b7fe700 1 -- 172.21.15.32:0/3844904422 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c06bce0 con 0x7f560c15be70 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560a7fc700 1 -- 172.21.15.32:0/3844904422 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58376081 0 0) 0x7f560c0b5ee0 con 0x7f560c0add40 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560b7fe700 1 --2- 172.21.15.32:0/3844904422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c15be70 0x7f560c0cfdf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f560c06c490 tx=0x7f55f4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd04b3c14b6c9c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560a7fc700 1 -- 172.21.15.32:0/3844904422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4011830 con 0x7f560c15be70 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560a7fc700 1 -- 172.21.15.32:0/3844904422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f4011990 con 0x7f560c15be70 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.552+0000 7f560a7fc700 1 -- 172.21.15.32:0/3844904422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4011d20 con 0x7f560c15be70 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 -- 172.21.15.32:0/3844904422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c15be70 msgr2=0x7f560c0cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 --2- 172.21.15.32:0/3844904422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c15be70 0x7f560c0cfdf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f560c06c490 tx=0x7f55f4009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 -- 172.21.15.32:0/3844904422 shutdown_connections 2023-11-24T07:58:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 --2- 172.21.15.32:0/3844904422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c15be70 0x7f560c0cfdf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 --2- 172.21.15.32:0/3844904422 >> v2:172.21.15.123:3300/0 conn(0x7f560c0746a0 0x7f560c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 -- 172.21.15.32:0/3844904422 >> 172.21.15.32:0/3844904422 conn(0x7f560c0d0bf0 msgr2=0x7f560c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 -- 172.21.15.32:0/3844904422 shutdown_connections 2023-11-24T07:58:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.553+0000 7f5612105700 1 -- 172.21.15.32:0/3844904422 wait complete. 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.554+0000 7f5612105700 1 Processor -- start 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.554+0000 7f5612105700 1 -- start start 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f560c0746a0 0x7f560c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 0x7f560c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c15be70 0x7f560c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c14b000 con 0x7f560c0add40 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15c570 con 0x7f560c15be70 2023-11-24T07:58:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15c850 con 0x7f560c0746a0 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 0x7f560c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 0x7f560c151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56218/0 (socket says 172.21.15.32:56218) 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 -- 172.21.15.32:0/2595666422 learned_addr learned my addr 172.21.15.32:0/2595666422 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560bfff700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c15be70 0x7f560c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560b7fe700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f560c0746a0 0x7f560c14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 -- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f560c0746a0 msgr2=0x7f560c14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f560c0746a0 0x7f560c14aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 -- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c15be70 msgr2=0x7f560c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c15be70 0x7f560c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 -- 172.21.15.32:0/2595666422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40094a0 con 0x7f560c0add40 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560bfff700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c15be70 0x7f560c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f560affd700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 0x7f560c151fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0011e0 tx=0x7f55fc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc004c80 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc004e00 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c15cb30 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.555+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc007310 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.556+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c15d0f0 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.556+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c068fb0 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.557+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f55fc007470 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.557+0000 7f5603fff700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f55ec06ae40 0x7f55ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.557+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f55fc088f50 con 0x7f560c0add40 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.558+0000 7f560b7fe700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f55ec06ae40 0x7f55ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.558+0000 7f560b7fe700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f55ec06ae40 0x7f55ec06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55f40090b0 tx=0x7f55f401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.562+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55fc054430 con 0x7f560c0add40 2023-11-24T07:58:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.763+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f560c159d20 con 0x7f560c0add40 2023-11-24T07:58:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.764+0000 7f5603fff700 1 -- 172.21.15.32:0/2595666422 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55fc057a50 con 0x7f560c0add40 2023-11-24T07:58:33.765 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f55ec06ae40 msgr2=0x7f55ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f55ec06ae40 0x7f55ec06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55f40090b0 tx=0x7f55f401c040 comp rx=0 tx=0).stop 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 msgr2=0x7f560c151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 0x7f560c151fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0011e0 tx=0x7f55fc00cae0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 shutdown_connections 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f55ec06ae40 0x7f55ec06d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f560c15be70 0x7f560c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f560c0746a0 0x7f560c14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 --2- 172.21.15.32:0/2595666422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560c0add40 0x7f560c151fd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 >> 172.21.15.32:0/2595666422 conn(0x7f560c0d0bf0 msgr2=0x7f560c0b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 shutdown_connections 2023-11-24T07:58:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.766+0000 7f5612105700 1 -- 172.21.15.32:0/2595666422 wait complete. 2023-11-24T07:58:33.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.902+0000 7f46b9328700 1 Processor -- start 2023-11-24T07:58:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.902+0000 7f46b9328700 1 -- start start 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b9328700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46b40b49a0 0x7f46b40d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b9328700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b415c4b0 con 0x7f46b406c710 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b9328700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b415c790 con 0x7f46b4152cc0 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b9328700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b415ca70 con 0x7f46b40b49a0 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b27fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46b406c710 0x7f46b40cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57938/0 (socket says 172.21.15.32:57938) 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b27fc700 1 -- 172.21.15.32:0/3732533507 learned_addr learned my addr 172.21.15.32:0/3732533507 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.903+0000 7f46b37fe700 1 --2- 172.21.15.32:0/3732533507 >> v2:172.21.15.32:3300/0 conn(0x7f46b40b49a0 0x7f46b40d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b37fe700 1 -- 172.21.15.32:0/3732533507 >> v1:172.21.15.123:6789/0 conn(0x7f46b4152cc0 legacy=0x7f46b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b37fe700 1 -- 172.21.15.32:0/3732533507 >> v1:172.21.15.32:6789/0 conn(0x7f46b406c710 legacy=0x7f46b40cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b1ffb700 1 -- 172.21.15.32:0/3732533507 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241275679 0 0) 0x7f46b415c4b0 con 0x7f46b406c710 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b37fe700 1 -- 172.21.15.32:0/3732533507 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b415cd50 con 0x7f46b40b49a0 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b37fe700 1 --2- 172.21.15.32:0/3732533507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b40d0630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009d90 tx=0x7f46ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bac19fa342f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b1ffb700 1 -- 172.21.15.32:0/3732533507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00e070 con 0x7f46b40b49a0 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b1ffb700 1 -- 172.21.15.32:0/3732533507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac00bbd0 con 0x7f46b40b49a0 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b1ffb700 1 -- 172.21.15.32:0/3732533507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac012430 con 0x7f46b40b49a0 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b9328700 1 -- 172.21.15.32:0/3732533507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 msgr2=0x7f46b40d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b9328700 1 --2- 172.21.15.32:0/3732533507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b40d0630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009d90 tx=0x7f46ac00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b9328700 1 -- 172.21.15.32:0/3732533507 shutdown_connections 2023-11-24T07:58:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.904+0000 7f46b9328700 1 --2- 172.21.15.32:0/3732533507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b40d0630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.905+0000 7f46b9328700 1 -- 172.21.15.32:0/3732533507 >> 172.21.15.32:0/3732533507 conn(0x7f46b4071b90 msgr2=0x7f46b4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.905+0000 7f46b9328700 1 -- 172.21.15.32:0/3732533507 shutdown_connections 2023-11-24T07:58:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.905+0000 7f46b9328700 1 -- 172.21.15.32:0/3732533507 wait complete. 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.906+0000 7f46b9328700 1 Processor -- start 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.906+0000 7f46b9328700 1 -- start start 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46b406c710 0x7f46b409d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b409d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4152cc0 0x7f46b4154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4155d50 con 0x7f46b40b49a0 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b413f6e0 con 0x7f46b4152cc0 2023-11-24T07:58:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b413f9c0 con 0x7f46b406c710 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b409d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b409d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56232/0 (socket says 172.21.15.32:56232) 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 -- 172.21.15.32:0/1774707282 learned_addr learned my addr 172.21.15.32:0/1774707282 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b2ffd700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46b406c710 0x7f46b409d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b37fe700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4152cc0 0x7f46b4154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 -- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46b406c710 msgr2=0x7f46b409d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46b406c710 0x7f46b409d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 -- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4152cc0 msgr2=0x7f46b4154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4152cc0 0x7f46b4154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 -- 172.21.15.32:0/1774707282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009a70 con 0x7f46b40b49a0 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b27fc700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b409d7c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46a400c520 tx=0x7f46a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4004c80 con 0x7f46b40b49a0 2023-11-24T07:58:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a4004e00 con 0x7f46b40b49a0 2023-11-24T07:58:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4007310 con 0x7f46b40b49a0 2023-11-24T07:58:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.907+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b413fca0 con 0x7f46b40b49a0 2023-11-24T07:58:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.908+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b409aa00 con 0x7f46b40b49a0 2023-11-24T07:58:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.909+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b40753f0 con 0x7f46b40b49a0 2023-11-24T07:58:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.909+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f46a4007470 con 0x7f46b40b49a0 2023-11-24T07:58:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.909+0000 7f469b7fe700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f46a006ae40 0x7f46a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.909+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f46a4088d60 con 0x7f46b40b49a0 2023-11-24T07:58:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.910+0000 7f46b2ffd700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f46a006ae40 0x7f46a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.910+0000 7f46b2ffd700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f46a006ae40 0x7f46a006d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46a8007fd0 tx=0x7f46a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:33.914+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46a408e020 con 0x7f46b40b49a0 2023-11-24T07:58:34.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.163+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46b4067f60 con 0x7f46b40b49a0 2023-11-24T07:58:34.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.163+0000 7f469b7fe700 1 -- 172.21.15.32:0/1774707282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f46a4054240 con 0x7f46b40b49a0 2023-11-24T07:58:34.165 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:34.165 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T07:58:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.165+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f46a006ae40 msgr2=0x7f46a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.165+0000 7f46b9328700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f46a006ae40 0x7f46a006d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46a8007fd0 tx=0x7f46a8009040 comp rx=0 tx=0).stop 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.165+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 msgr2=0x7f46b409d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.165+0000 7f46b9328700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b409d7c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46a400c520 tx=0x7f46a400c830 comp rx=0 tx=0).stop 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 shutdown_connections 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f46a006ae40 0x7f46a006d300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4152cc0 0x7f46b4154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46b406c710 0x7f46b409d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 --2- 172.21.15.32:0/1774707282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b40b49a0 0x7f46b409d7c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 >> 172.21.15.32:0/1774707282 conn(0x7f46b4071b90 msgr2=0x7f46b40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 shutdown_connections 2023-11-24T07:58:34.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.166+0000 7f46b9328700 1 -- 172.21.15.32:0/1774707282 wait complete. 2023-11-24T07:58:34.169 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2023-11-24T07:58:34.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.303+0000 7f392d6c7700 1 Processor -- start 2023-11-24T07:58:34.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.303+0000 7f392d6c7700 1 -- start start 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.303+0000 7f392d6c7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f392806c710 0x7f39280b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.303+0000 7f392d6c7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39280ae430 con 0x7f392815e080 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.303+0000 7f392d6c7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39280ae710 con 0x7f39280af4d0 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.303+0000 7f392d6c7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39280af0b0 con 0x7f392806c710 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f39267fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f392806c710 0x7f39280b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f3926ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f392815e080 0x7f392806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57952/0 (socket says 172.21.15.32:57952) 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f39267fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f392806c710 0x7f39280b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56244/0 (socket says 172.21.15.32:56244) 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f3926ffd700 1 -- 172.21.15.32:0/2084891901 learned_addr learned my addr 172.21.15.32:0/2084891901 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f39267fc700 1 -- 172.21.15.32:0/2084891901 >> v1:172.21.15.123:6789/0 conn(0x7f39280af4d0 legacy=0x7f392806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f39267fc700 1 -- 172.21.15.32:0/2084891901 >> v1:172.21.15.32:6789/0 conn(0x7f392815e080 legacy=0x7f392806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f39267fc700 1 -- 172.21.15.32:0/2084891901 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280ae9f0 con 0x7f392806c710 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f3925ffb700 1 -- 172.21.15.32:0/2084891901 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678741332 0 0) 0x7f39280ae430 con 0x7f392815e080 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f39267fc700 1 --2- 172.21.15.32:0/2084891901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392806c710 0x7f39280b5050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3910009840 tx=0x7f3910009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3df40e09cf88de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f3925ffb700 1 -- 172.21.15.32:0/2084891901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910011830 con 0x7f392806c710 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f3925ffb700 1 -- 172.21.15.32:0/2084891901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910011990 con 0x7f392806c710 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.304+0000 7f3925ffb700 1 -- 172.21.15.32:0/2084891901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910011d20 con 0x7f392806c710 2023-11-24T07:58:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 -- 172.21.15.32:0/2084891901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392806c710 msgr2=0x7f39280b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2084891901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392806c710 0x7f39280b5050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3910009840 tx=0x7f3910009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 -- 172.21.15.32:0/2084891901 shutdown_connections 2023-11-24T07:58:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2084891901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392806c710 0x7f39280b5050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 -- 172.21.15.32:0/2084891901 >> 172.21.15.32:0/2084891901 conn(0x7f3928071b90 msgr2=0x7f3928071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 -- 172.21.15.32:0/2084891901 shutdown_connections 2023-11-24T07:58:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.305+0000 7f392d6c7700 1 -- 172.21.15.32:0/2084891901 wait complete. 2023-11-24T07:58:34.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.306+0000 7f392d6c7700 1 Processor -- start 2023-11-24T07:58:34.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.306+0000 7f392d6c7700 1 -- start start 2023-11-24T07:58:34.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f392d6c7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f392806c710 0x7f39280ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f392d6c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 0x7f3928160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f392d6c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392815e080 0x7f3928144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f392d6c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280cf8f0 con 0x7f39280af4d0 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f392d6c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39281613f0 con 0x7f392815e080 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f392d6c7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39280cece0 con 0x7f392806c710 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 0x7f3928160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 0x7f3928160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56248/0 (socket says 172.21.15.32:56248) 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 -- 172.21.15.32:0/2283394421 learned_addr learned my addr 172.21.15.32:0/2283394421 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:34.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 -- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f392806c710 msgr2=0x7f39280ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39277fe700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392815e080 0x7f3928144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f392806c710 0x7f39280ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 -- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392815e080 msgr2=0x7f3928144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392815e080 0x7f3928144220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 -- 172.21.15.32:0/2283394421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100094a0 con 0x7f39280af4d0 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.307+0000 7f39267fc700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 0x7f3928160eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39100090b0 tx=0x7f3910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.308+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391001a900 con 0x7f39280af4d0 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.308+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391001aa60 con 0x7f39280af4d0 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.308+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39280cefc0 con 0x7f39280af4d0 2023-11-24T07:58:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.308+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391001add0 con 0x7f39280af4d0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.308+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39280cf580 con 0x7f39280af4d0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.308+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39280753f0 con 0x7f39280af4d0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.309+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f3910022070 con 0x7f39280af4d0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.309+0000 7f391f7fe700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f390806ae40 0x7f390806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.310+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f391008a390 con 0x7f39280af4d0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.310+0000 7f3926ffd700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f390806ae40 0x7f390806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.310+0000 7f3926ffd700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f390806ae40 0x7f390806d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f3918007a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.314+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39100559b0 con 0x7f39280af4d0 2023-11-24T07:58:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.517+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39280cfa30 con 0x7f39280af4d0 2023-11-24T07:58:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.518+0000 7f391f7fe700 1 -- 172.21.15.32:0/2283394421 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12851 (secure 0 0 0) 0x7f39280cfa30 con 0x7f39280af4d0 2023-11-24T07:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:33.991175+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f390806ae40 msgr2=0x7f390806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f390806ae40 0x7f390806d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f3918007a50 comp rx=0 tx=0).stop 2023-11-24T07:58:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 msgr2=0x7f3928160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 0x7f3928160eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39100090b0 tx=0x7f3910004750 comp rx=0 tx=0).stop 2023-11-24T07:58:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 shutdown_connections 2023-11-24T07:58:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f390806ae40 0x7f390806d300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392815e080 0x7f3928144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f392806c710 0x7f39280ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 --2- 172.21.15.32:0/2283394421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39280af4d0 0x7f3928160eb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 >> 172.21.15.32:0/2283394421 conn(0x7f3928071b90 msgr2=0x7f392806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 shutdown_connections 2023-11-24T07:58:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.520+0000 7f392d6c7700 1 -- 172.21.15.32:0/2283394421 wait complete. 2023-11-24T07:58:34.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:34.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.663+0000 7f23f59b6700 1 Processor -- start 2023-11-24T07:58:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.663+0000 7f23f59b6700 1 -- start start 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23f59b6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23f015b370 0x7f23f015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23f59b6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f00b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23f59b6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f00ae5e0 con 0x7f23f01587f0 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23f59b6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f00b4630 con 0x7f23f015b370 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23f59b6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f015be30 con 0x7f23f00b3c60 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23ee7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23f015b370 0x7f23f015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23ee7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23f015b370 0x7f23f015b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56256/0 (socket says 172.21.15.32:56256) 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23ee7fc700 1 -- 172.21.15.32:0/662588724 learned_addr learned my addr 172.21.15.32:0/662588724 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.664+0000 7f23ef7fe700 1 --2- 172.21.15.32:0/662588724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f00b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23ee7fc700 1 -- 172.21.15.32:0/662588724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 msgr2=0x7f23f00b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23ee7fc700 1 --2- 172.21.15.32:0/662588724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f00b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23ee7fc700 1 -- 172.21.15.32:0/662588724 >> v1:172.21.15.123:6789/0 conn(0x7f23f01587f0 legacy=0x7f23f006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23ee7fc700 1 -- 172.21.15.32:0/662588724 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f015c110 con 0x7f23f015b370 2023-11-24T07:58:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23ee7fc700 1 --2- 172.21.15.32:0/662588724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f015b800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009920 tx=0x7f23e4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219e9c5461d8d210 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23edffb700 1 -- 172.21.15.32:0/662588724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011830 con 0x7f23f015b370 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23edffb700 1 -- 172.21.15.32:0/662588724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4011990 con 0x7f23f015b370 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.665+0000 7f23edffb700 1 -- 172.21.15.32:0/662588724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011d20 con 0x7f23f015b370 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 -- 172.21.15.32:0/662588724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 msgr2=0x7f23f015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 --2- 172.21.15.32:0/662588724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f015b800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009920 tx=0x7f23e4009c30 comp rx=0 tx=0).stop 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 -- 172.21.15.32:0/662588724 shutdown_connections 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 --2- 172.21.15.32:0/662588724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f00b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 --2- 172.21.15.32:0/662588724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f015b800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 -- 172.21.15.32:0/662588724 >> 172.21.15.32:0/662588724 conn(0x7f23f00d0bf0 msgr2=0x7f23f00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 -- 172.21.15.32:0/662588724 shutdown_connections 2023-11-24T07:58:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.666+0000 7f23f59b6700 1 -- 172.21.15.32:0/662588724 wait complete. 2023-11-24T07:58:34.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.667+0000 7f23f59b6700 1 Processor -- start 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.667+0000 7f23f59b6700 1 -- start start 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23f59b6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f0156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23f59b6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23f01587f0 0x7f23f01e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23f59b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f01e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23f59b6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f0156590 con 0x7f23f015b370 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23f59b6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f015d270 con 0x7f23f01587f0 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23f59b6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f015d550 con 0x7f23f00b3c60 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f01e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f01e64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56264/0 (socket says 172.21.15.32:56264) 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 -- 172.21.15.32:0/1556416105 learned_addr learned my addr 172.21.15.32:0/1556416105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ee7fc700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23f01587f0 0x7f23f01e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23eeffd700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f0156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 -- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 msgr2=0x7f23f0156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f0156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 -- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23f01587f0 msgr2=0x7f23f01e5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23f01587f0 0x7f23f01e5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ef7fe700 1 -- 172.21.15.32:0/1556416105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4009550 con 0x7f23f015b370 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23eeffd700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f0156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.668+0000 7f23ee7fc700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23f01587f0 0x7f23f01e5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.669+0000 7f23ef7fe700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f01e64c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23e800efc0 tx=0x7f23e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.669+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e800cd40 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.669+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e800f040 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.669+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f015d830 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.669+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e80146c0 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.669+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f015ddf0 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.670+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f0068fb0 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.670+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f23e8014820 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.671+0000 7f23d7fff700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f23dc06ae40 0x7f23dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.671+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23e8089ab0 con 0x7f23f015b370 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.671+0000 7f23eeffd700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f23dc06ae40 0x7f23dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.672+0000 7f23eeffd700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f23dc06ae40 0x7f23dc06d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23f0068340 tx=0x7f23e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.676+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23e8054f90 con 0x7f23f015b370 2023-11-24T07:58:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.887+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23f00b5300 con 0x7f23f015b370 2023-11-24T07:58:34.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.888+0000 7f23d7fff700 1 -- 172.21.15.32:0/1556416105 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23e80180a0 con 0x7f23f015b370 2023-11-24T07:58:34.890 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:58:34.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f23dc06ae40 msgr2=0x7f23dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f23dc06ae40 0x7f23dc06d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23f0068340 tx=0x7f23e0009040 comp rx=0 tx=0).stop 2023-11-24T07:58:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 msgr2=0x7f23f01e64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f01e64c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23e800efc0 tx=0x7f23e800c3a0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 shutdown_connections 2023-11-24T07:58:34.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f23dc06ae40 0x7f23dc06d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f015b370 0x7f23f01e64c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23f00b3c60 0x7f23f0156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 --2- 172.21.15.32:0/1556416105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23f01587f0 0x7f23f01e5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 >> 172.21.15.32:0/1556416105 conn(0x7f23f00d0bf0 msgr2=0x7f23f00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 shutdown_connections 2023-11-24T07:58:34.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:34.890+0000 7f23f59b6700 1 -- 172.21.15.32:0/1556416105 wait complete. 2023-11-24T07:58:34.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:58:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.031+0000 7f69d1a6d700 1 Processor -- start 2023-11-24T07:58:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.031+0000 7f69d1a6d700 1 -- start start 2023-11-24T07:58:35.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69d1a6d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc15af10 0x7f69cc15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69d1a6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69cc0ce140 0x7f69cc15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69d1a6d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc1414c0 con 0x7f69cc15f460 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69d1a6d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc15f8c0 con 0x7f69cc0ce140 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69d1a6d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0ce9f0 con 0x7f69cc15af10 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69cc0ce140 0x7f69cc15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69cc0ce140 0x7f69cc15ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56270/0 (socket says 172.21.15.32:56270) 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 -- 172.21.15.32:0/3741256301 learned_addr learned my addr 172.21.15.32:0/3741256301 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69cbfff700 1 --1- 172.21.15.32:0/3741256301 >> v1:172.21.15.32:6789/0 conn(0x7f69cc15f460 0x7f69cc15b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57984/0 (socket says 172.21.15.32:57984) 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69cb7fe700 1 --2- 172.21.15.32:0/3741256301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc15af10 0x7f69cc15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 -- 172.21.15.32:0/3741256301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc15af10 msgr2=0x7f69cc15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 --2- 172.21.15.32:0/3741256301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc15af10 0x7f69cc15e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 -- 172.21.15.32:0/3741256301 >> v1:172.21.15.32:6789/0 conn(0x7f69cc15f460 legacy=0x7f69cc15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 -- 172.21.15.32:0/3741256301 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0cecd0 con 0x7f69cc0ce140 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69ca7fc700 1 -- 172.21.15.32:0/3741256301 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282560315 0 0) 0x7f69cc1414c0 con 0x7f69cc15f460 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.032+0000 7f69caffd700 1 --2- 172.21.15.32:0/3741256301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc0ce140 0x7f69cc15ef20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69c00098f0 tx=0x7f69c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e78e1268a5a1aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69ca7fc700 1 -- 172.21.15.32:0/3741256301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0011830 con 0x7f69cc0ce140 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69ca7fc700 1 -- 172.21.15.32:0/3741256301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0011990 con 0x7f69cc0ce140 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69ca7fc700 1 -- 172.21.15.32:0/3741256301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0011d20 con 0x7f69cc0ce140 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69d1a6d700 1 -- 172.21.15.32:0/3741256301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc0ce140 msgr2=0x7f69cc15ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/3741256301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc0ce140 0x7f69cc15ef20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69c00098f0 tx=0x7f69c0009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:35.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69d1a6d700 1 -- 172.21.15.32:0/3741256301 shutdown_connections 2023-11-24T07:58:35.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/3741256301 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc15af10 0x7f69cc15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/3741256301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc0ce140 0x7f69cc15ef20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.033+0000 7f69d1a6d700 1 -- 172.21.15.32:0/3741256301 >> 172.21.15.32:0/3741256301 conn(0x7f69cc0cd570 msgr2=0x7f69cc0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:35.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.034+0000 7f69d1a6d700 1 -- 172.21.15.32:0/3741256301 shutdown_connections 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.034+0000 7f69d1a6d700 1 -- 172.21.15.32:0/3741256301 wait complete. 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.035+0000 7f69d1a6d700 1 Processor -- start 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.035+0000 7f69d1a6d700 1 -- start start 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69d1a6d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc0ce140 0x7f69cc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69d1a6d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15af10 0x7f69cc156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69d1a6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 0x7f69cc0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69d1a6d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b42e0 con 0x7f69cc15f460 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69d1a6d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b4530 con 0x7f69cc15af10 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69d1a6d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b4810 con 0x7f69cc0ce140 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 0x7f69cc0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 0x7f69cc0b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56278/0 (socket says 172.21.15.32:56278) 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 -- 172.21.15.32:0/1097923940 learned_addr learned my addr 172.21.15.32:0/1097923940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cb7fe700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc0ce140 0x7f69cc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69caffd700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15af10 0x7f69cc156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 -- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc0ce140 msgr2=0x7f69cc0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc0ce140 0x7f69cc0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 -- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15af10 msgr2=0x7f69cc156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15af10 0x7f69cc156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cbfff700 1 -- 172.21.15.32:0/1097923940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c0009550 con 0x7f69cc15f460 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69caffd700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15af10 0x7f69cc156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.036+0000 7f69cb7fe700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc0ce140 0x7f69cc0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.037+0000 7f69cbfff700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 0x7f69cc0b0fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69c400cb60 tx=0x7f69c400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.037+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400ab50 con 0x7f69cc15f460 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.037+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c400acb0 con 0x7f69cc15f460 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.037+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc0b4af0 con 0x7f69cc15f460 2023-11-24T07:58:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.037+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4007310 con 0x7f69cc15f460 2023-11-24T07:58:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.037+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc0b50b0 con 0x7f69cc15f460 2023-11-24T07:58:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.038+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc067f60 con 0x7f69cc15f460 2023-11-24T07:58:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.038+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f69c4007470 con 0x7f69cc15f460 2023-11-24T07:58:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.039+0000 7f69b3fff700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f69b806ae40 0x7f69b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.039+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69c4088f00 con 0x7f69cc15f460 2023-11-24T07:58:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.039+0000 7f69cb7fe700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f69b806ae40 0x7f69b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.039+0000 7f69cb7fe700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f69b806ae40 0x7f69b806d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009c30 tx=0x7f69bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.043+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69c4054470 con 0x7f69cc15f460 2023-11-24T07:58:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.256+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f69cc072e00 con 0x7f69cc15f460 2023-11-24T07:58:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.257+0000 7f69b3fff700 1 -- 172.21.15.32:0/1097923940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f69c4018090 con 0x7f69cc15f460 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:58:35.259 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-11-24T07:58:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f69b806ae40 msgr2=0x7f69b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f69b806ae40 0x7f69b806d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009c30 tx=0x7f69bc008040 comp rx=0 tx=0).stop 2023-11-24T07:58:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 msgr2=0x7f69cc0b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 0x7f69cc0b0fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69c400cb60 tx=0x7f69c400cf20 comp rx=0 tx=0).stop 2023-11-24T07:58:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 shutdown_connections 2023-11-24T07:58:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f69b806ae40 0x7f69b806d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69cc15f460 0x7f69cc0b0fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69cc0ce140 0x7f69cc0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 --2- 172.21.15.32:0/1097923940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69cc15af10 0x7f69cc156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 >> 172.21.15.32:0/1097923940 conn(0x7f69cc0cd570 msgr2=0x7f69cc06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 shutdown_connections 2023-11-24T07:58:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.259+0000 7f69d1a6d700 1 -- 172.21.15.32:0/1097923940 wait complete. 2023-11-24T07:58:35.273 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:58:35.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:58:35.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.400+0000 7f888ac8b700 1 Processor -- start 2023-11-24T07:58:35.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.401+0000 7f888ac8b700 1 -- start start 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.401+0000 7f888ac8b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88840b0450 0x7f88840d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.401+0000 7f888ac8b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840ae9f0 con 0x7f88840cfff0 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.401+0000 7f888ac8b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840aefb0 con 0x7f888415be40 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.401+0000 7f888ac8b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840d0970 con 0x7f88840b0450 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8883fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88840cfff0 0x7f88840ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57994/0 (socket says 172.21.15.32:57994) 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8883fff700 1 -- 172.21.15.32:0/3521046002 learned_addr learned my addr 172.21.15.32:0/3521046002 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:35.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8889228700 1 --2- 172.21.15.32:0/3521046002 >> v2:172.21.15.123:3300/0 conn(0x7f88840b0450 0x7f88840d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f88837fe700 1 -- 172.21.15.32:0/3521046002 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657601677 0 0) 0x7f88840aefb0 con 0x7f888415be40 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f88837fe700 1 -- 172.21.15.32:0/3521046002 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88740035b0 con 0x7f888415be40 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8889228700 1 -- 172.21.15.32:0/3521046002 >> v1:172.21.15.123:6789/0 conn(0x7f888415be40 legacy=0x7f88840b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8889228700 1 -- 172.21.15.32:0/3521046002 >> v1:172.21.15.32:6789/0 conn(0x7f88840cfff0 legacy=0x7f88840ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8889228700 1 -- 172.21.15.32:0/3521046002 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840b3120 con 0x7f88840b0450 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f8889228700 1 --2- 172.21.15.32:0/3521046002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f88840d0430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f887c009d90 tx=0x7f887c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7ea0f6bcfb6aee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.402+0000 7f88837fe700 1 -- 172.21.15.32:0/3521046002 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c00e070 con 0x7f88840b0450 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f88837fe700 1 -- 172.21.15.32:0/3521046002 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c00bb20 con 0x7f88840b0450 2023-11-24T07:58:35.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f88837fe700 1 -- 172.21.15.32:0/3521046002 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c012430 con 0x7f88840b0450 2023-11-24T07:58:35.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 -- 172.21.15.32:0/3521046002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 msgr2=0x7f88840d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3521046002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f88840d0430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f887c009d90 tx=0x7f887c00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 -- 172.21.15.32:0/3521046002 shutdown_connections 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3521046002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f88840d0430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 -- 172.21.15.32:0/3521046002 >> 172.21.15.32:0/3521046002 conn(0x7f888409aa00 msgr2=0x7f888409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 -- 172.21.15.32:0/3521046002 shutdown_connections 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.403+0000 7f888ac8b700 1 -- 172.21.15.32:0/3521046002 wait complete. 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.405+0000 7f888ac8b700 1 Processor -- start 2023-11-24T07:58:35.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.405+0000 7f888ac8b700 1 -- start start 2023-11-24T07:58:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f888ac8b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f888406c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f888ac8b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88840cfff0 0x7f888414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f888ac8b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888415be40 0x7f888406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f888ac8b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840ccf80 con 0x7f88840cfff0 2023-11-24T07:58:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f888ac8b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8884150330 con 0x7f88840b0450 2023-11-24T07:58:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f888ac8b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888406bf40 con 0x7f888415be40 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8883fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88840cfff0 0x7f888414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8883fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88840cfff0 0x7f888414fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56290/0 (socket says 172.21.15.32:56290) 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8883fff700 1 -- 172.21.15.32:0/3302674478 learned_addr learned my addr 172.21.15.32:0/3302674478 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8888a27700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f888406c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8889228700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888415be40 0x7f888406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8889228700 1 -- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 msgr2=0x7f888406c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8889228700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f888406c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8889228700 1 -- 172.21.15.32:0/3302674478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88840cfff0 msgr2=0x7f888414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8889228700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88840cfff0 0x7f888414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.406+0000 7f8889228700 1 -- 172.21.15.32:0/3302674478 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c009a70 con 0x7f888415be40 2023-11-24T07:58:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.407+0000 7f8889228700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888415be40 0x7f888406ba00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f887c016000 tx=0x7f887c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.407+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c012430 con 0x7f888415be40 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.407+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c020070 con 0x7f888415be40 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.407+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88840cc5e0 con 0x7f888415be40 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.407+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c004280 con 0x7f888415be40 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.407+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88840ccba0 con 0x7f888415be40 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.408+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8884067f60 con 0x7f888415be40 2023-11-24T07:58:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.408+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f887c0043e0 con 0x7f888415be40 2023-11-24T07:58:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.409+0000 7f88817fa700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f886806ae40 0x7f886806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.409+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f887c022070 con 0x7f888415be40 2023-11-24T07:58:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.409+0000 7f8888a27700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f886806ae40 0x7f886806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.409+0000 7f8888a27700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f886806ae40 0x7f886806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88780011e0 tx=0x7f8878009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.413+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f887c056220 con 0x7f888415be40 2023-11-24T07:58:35.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:35.617+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f88840cd0c0 con 0x7f888415be40 2023-11-24T07:58:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.634+0000 7f88817fa700 1 -- 172.21.15.32:0/3302674478 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f88840cd0c0 con 0x7f888415be40 2023-11-24T07:58:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.638+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f886806ae40 msgr2=0x7f886806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.638+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f886806ae40 0x7f886806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88780011e0 tx=0x7f8878009040 comp rx=0 tx=0).stop 2023-11-24T07:58:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.638+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888415be40 msgr2=0x7f888406ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.638+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888415be40 0x7f888406ba00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f887c016000 tx=0x7f887c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.639+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 shutdown_connections 2023-11-24T07:58:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.639+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f886806ae40 0x7f886806d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.639+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888415be40 0x7f888406ba00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.639+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88840b0450 0x7f888406c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.639+0000 7f888ac8b700 1 --2- 172.21.15.32:0/3302674478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88840cfff0 0x7f888414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.639+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 >> 172.21.15.32:0/3302674478 conn(0x7f888409aa00 msgr2=0x7f888406f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.640+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 shutdown_connections 2023-11-24T07:58:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.640+0000 7f888ac8b700 1 -- 172.21.15.32:0/3302674478 wait complete. 2023-11-24T07:58:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-11-24T07:58:36.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:58:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.797+0000 7f590bd7d700 1 Processor -- start 2023-11-24T07:58:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.797+0000 7f590bd7d700 1 -- start start 2023-11-24T07:58:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.798+0000 7f590bd7d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59040c6c30 0x7f59040cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.798+0000 7f590bd7d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59040d0140 con 0x7f59040cef40 2023-11-24T07:58:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.798+0000 7f590bd7d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59040cf3d0 con 0x7f590415bf50 2023-11-24T07:58:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.798+0000 7f590bd7d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590415c330 con 0x7f59040c6c30 2023-11-24T07:58:36.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.798+0000 7f590a31a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f59040cef40 0x7f59040cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58012/0 (socket says 172.21.15.32:58012) 2023-11-24T07:58:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.798+0000 7f590a31a700 1 -- 172.21.15.32:0/3093401833 learned_addr learned my addr 172.21.15.32:0/3093401833 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5909318700 1 --2- 172.21.15.32:0/3093401833 >> v2:172.21.15.123:3300/0 conn(0x7f59040c6c30 0x7f59040cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003588566 0 0) 0x7f59040d0140 con 0x7f59040cef40 2023-11-24T07:58:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f00035b0 con 0x7f59040cef40 2023-11-24T07:58:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1479637842 0 0) 0x7f58f00035b0 con 0x7f59040cef40 2023-11-24T07:58:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 >> v2:172.21.15.123:3300/0 conn(0x7f59040c6c30 msgr2=0x7f59040cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 --2- 172.21.15.32:0/3093401833 >> v2:172.21.15.123:3300/0 conn(0x7f59040c6c30 0x7f59040cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 >> v1:172.21.15.123:6789/0 conn(0x7f590415bf50 legacy=0x7f59040ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040d03e0 con 0x7f59040cef40 2023-11-24T07:58:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f58fc0019f0 con 0x7f59040cef40 2023-11-24T07:58:36.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 >> v1:172.21.15.32:6789/0 conn(0x7f59040cef40 legacy=0x7f59040cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f0003a00 0x7f58f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0004850 0x7f58f0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005210 0x7f58f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590415c330 con 0x7f58f0005210 2023-11-24T07:58:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f000c0b0 con 0x7f58f0004850 2023-11-24T07:58:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f000c270 con 0x7f58f0003a00 2023-11-24T07:58:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f590a31a700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005210 0x7f58f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5909318700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0004850 0x7f58f0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f5909b19700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f0003a00 0x7f58f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f590a31a700 1 -- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f0003a00 msgr2=0x7f58f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f590a31a700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f58f0003a00 0x7f58f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f590a31a700 1 -- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0004850 msgr2=0x7f58f0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f590a31a700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0004850 0x7f58f0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.799+0000 7f590a31a700 1 -- 172.21.15.32:0/3093401833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00035b0 con 0x7f58f0005210 2023-11-24T07:58:36.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590a31a700 1 -- 172.21.15.32:0/3093401833 reap_dead start 2023-11-24T07:58:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590a31a700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005210 0x7f58f000ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00a830 tx=0x7f58f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3f93911ee47923 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00a2f0 con 0x7f58f0005210 2023-11-24T07:58:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc012600 con 0x7f58f0005210 2023-11-24T07:58:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f5908b17700 1 -- 172.21.15.32:0/3093401833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0129b0 con 0x7f58f0005210 2023-11-24T07:58:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 -- 172.21.15.32:0/3093401833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005210 msgr2=0x7f58f000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005210 0x7f58f000ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00a830 tx=0x7f58f0005880 comp rx=0 tx=0).stop 2023-11-24T07:58:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 -- 172.21.15.32:0/3093401833 shutdown_connections 2023-11-24T07:58:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 --2- 172.21.15.32:0/3093401833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f0005210 0x7f58f000ba10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 -- 172.21.15.32:0/3093401833 >> 172.21.15.32:0/3093401833 conn(0x7f59040b3780 msgr2=0x7f59040b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 -- 172.21.15.32:0/3093401833 shutdown_connections 2023-11-24T07:58:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.800+0000 7f590bd7d700 1 -- 172.21.15.32:0/3093401833 wait complete. 2023-11-24T07:58:36.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 Processor -- start 2023-11-24T07:58:36.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 -- start start 2023-11-24T07:58:36.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0005210 0x7f5904151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 0x7f59040d2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59040b5280 0x7f59040b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5904152460 con 0x7f59040d20c0 2023-11-24T07:58:36.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590415e5c0 con 0x7f58f0005210 2023-11-24T07:58:36.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f590bd7d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59040d2a80 con 0x7f59040b5280 2023-11-24T07:58:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.802+0000 7f5909318700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 0x7f59040d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 0x7f59040d2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56310/0 (socket says 172.21.15.32:56310) 2023-11-24T07:58:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 -- 172.21.15.32:0/1678563166 learned_addr learned my addr 172.21.15.32:0/1678563166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909b19700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0005210 0x7f5904151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f590a31a700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59040b5280 0x7f59040b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 -- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59040b5280 msgr2=0x7f59040b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59040b5280 0x7f59040b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 -- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0005210 msgr2=0x7f5904151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0005210 0x7f5904151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 -- 172.21.15.32:0/1678563166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040b5c50 con 0x7f59040d20c0 2023-11-24T07:58:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f5909318700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 0x7f59040d2540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58f40119a0 tx=0x7f58f4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400f780 con 0x7f59040d20c0 2023-11-24T07:58:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400f8e0 con 0x7f59040d20c0 2023-11-24T07:58:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59040b5e90 con 0x7f59040d20c0 2023-11-24T07:58:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400fc70 con 0x7f59040d20c0 2023-11-24T07:58:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.803+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59040b6730 con 0x7f59040d20c0 2023-11-24T07:58:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.804+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5904068fb0 con 0x7f59040d20c0 2023-11-24T07:58:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.805+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58f4014440 con 0x7f59040d20c0 2023-11-24T07:58:36.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.805+0000 7f58fa7fc700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f58e806aed0 0x7f58e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.805+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f58f40897c0 con 0x7f59040d20c0 2023-11-24T07:58:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.805+0000 7f5909b19700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f58e806aed0 0x7f58e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.806+0000 7f5909b19700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f58e806aed0 0x7f58e806d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5900000c00 tx=0x7f5900009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:36.810+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58f4054ae0 con 0x7f59040d20c0 2023-11-24T07:58:37.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.012+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f59040b5c50 con 0x7f59040d20c0 2023-11-24T07:58:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.636+0000 7f58fa7fc700 1 -- 172.21.15.32:0/1678563166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f59040b5c50 con 0x7f59040d20c0 2023-11-24T07:58:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f58e806aed0 msgr2=0x7f58e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f58e806aed0 0x7f58e806d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5900000c00 tx=0x7f5900009040 comp rx=0 tx=0).stop 2023-11-24T07:58:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 msgr2=0x7f59040d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 0x7f59040d2540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58f40119a0 tx=0x7f58f4011cb0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 shutdown_connections 2023-11-24T07:58:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f58e806aed0 0x7f58e806d390 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5900000c00 tx=0x7f5900009040 comp rx=0 tx=0).stop 2023-11-24T07:58:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59040b5280 0x7f59040b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58f0005210 0x7f5904151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 --2- 172.21.15.32:0/1678563166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59040d20c0 0x7f59040d2540 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 >> 172.21.15.32:0/1678563166 conn(0x7f59040b3780 msgr2=0x7f59040cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.640+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 shutdown_connections 2023-11-24T07:58:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.641+0000 7f590bd7d700 1 -- 172.21.15.32:0/1678563166 wait complete. 2023-11-24T07:58:37.658 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-11-24T07:58:37.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:58:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f4345b70700 1 Processor -- start 2023-11-24T07:58:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f4345b70700 1 -- start start 2023-11-24T07:58:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f4345b70700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400d39d0 0x7f43400b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f4345b70700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43400d2930 con 0x7f434015e140 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f4345b70700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43400d2c10 con 0x7f43400d45d0 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f4345b70700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434015e520 con 0x7f43400d39d0 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f433f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f434015e140 0x7f43400d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58028/0 (socket says 172.21.15.32:58028) 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.797+0000 7f433f7fe700 1 -- 172.21.15.32:0/1182041717 learned_addr learned my addr 172.21.15.32:0/1182041717 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433ffff700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400d39d0 0x7f43400b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946166066 0 0) 0x7f43400d2930 con 0x7f434015e140 2023-11-24T07:58:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43280035b0 con 0x7f434015e140 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300606183 0 0) 0x7f43400d2c10 con 0x7f43400d45d0 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43400d2930 con 0x7f43400d45d0 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2957131904 0 0) 0x7f43280035b0 con 0x7f434015e140 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400d39d0 msgr2=0x7f43400b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400d39d0 0x7f43400b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 >> v1:172.21.15.123:6789/0 conn(0x7f43400d45d0 legacy=0x7f43400b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400d2ef0 con 0x7f434015e140 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f4330001670 con 0x7f434015e140 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 >> v1:172.21.15.32:6789/0 conn(0x7f434015e140 legacy=0x7f43400d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4328003bc0 0x7f4328003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f4328004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328005210 0x7f432800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434015e520 con 0x7f4328004850 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432800c000 con 0x7f4328005210 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432800c1c0 con 0x7f4328003bc0 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433ffff700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400d39d0 0x7f43400b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433effd700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f4328004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433f7fe700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4328003bc0 0x7f4328003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433ffff700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328005210 0x7f432800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433effd700 1 -- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4328003bc0 msgr2=0x7f4328003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433effd700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4328003bc0 0x7f4328003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433effd700 1 -- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328005210 msgr2=0x7f432800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433effd700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328005210 0x7f432800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.798+0000 7f433effd700 1 -- 172.21.15.32:0/1182041717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280035b0 con 0x7f4328004850 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f433effd700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f4328004cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f433400c320 tx=0x7f4328005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e675c7ba6343a84b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334001f20 con 0x7f4328004850 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4334003a00 con 0x7f4328004850 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f433e7fc700 1 -- 172.21.15.32:0/1182041717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433400cea0 con 0x7f4328004850 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f4345b70700 1 -- 172.21.15.32:0/1182041717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 msgr2=0x7f4328004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f4345b70700 1 --2- 172.21.15.32:0/1182041717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f4328004cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f433400c320 tx=0x7f4328005880 comp rx=0 tx=0).stop 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f433ffff700 1 -- 172.21.15.32:0/1182041717 reap_dead start 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f4345b70700 1 -- 172.21.15.32:0/1182041717 shutdown_connections 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f4345b70700 1 -- 172.21.15.32:0/1182041717 >> 172.21.15.32:0/1182041717 conn(0x7f434006b410 msgr2=0x7f434006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f4345b70700 1 -- 172.21.15.32:0/1182041717 shutdown_connections 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.799+0000 7f4345b70700 1 -- 172.21.15.32:0/1182041717 wait complete. 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.801+0000 7f4345b70700 1 Processor -- start 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.801+0000 7f4345b70700 1 -- start start 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.801+0000 7f4345b70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f43401535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43400b3c10 0x7f43400ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400b4cf0 0x7f43400b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4340153b00 con 0x7f4328004850 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400c90e0 con 0x7f43400b3c10 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400c93c0 con 0x7f43400b4cf0 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f43401535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f43401535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56328/0 (socket says 172.21.15.32:56328) 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 -- 172.21.15.32:0/4006564674 learned_addr learned my addr 172.21.15.32:0/4006564674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433effd700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43400b3c10 0x7f43400ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433ffff700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400b4cf0 0x7f43400b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 -- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400b4cf0 msgr2=0x7f43400b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400b4cf0 0x7f43400b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 -- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43400b3c10 msgr2=0x7f43400ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43400b3c10 0x7f43400ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 -- 172.21.15.32:0/4006564674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400c96a0 con 0x7f4328004850 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f433f7fe700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f43401535c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4330007340 tx=0x7f4330007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000f660 con 0x7f4328004850 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433000f7e0 con 0x7f4328004850 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000fb90 con 0x7f4328004850 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43400c9980 con 0x7f4328004850 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.802+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43400ca6c0 con 0x7f4328004850 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.803+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4340068fb0 con 0x7f4328004850 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.804+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f433000fcf0 con 0x7f4328004850 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.804+0000 7f4327fff700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f432c06aed0 0x7f432c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.804+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f43300893f0 con 0x7f4328004850 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.804+0000 7f433effd700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f432c06aed0 0x7f432c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.806+0000 7f433effd700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f432c06aed0 0x7f432c06d390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4334005040 tx=0x7f4334009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:37.810+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4330054590 con 0x7f4328004850 2023-11-24T07:58:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.059+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f43400b9870 con 0x7f4328004850 2023-11-24T07:58:38.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.648+0000 7f4327fff700 1 -- 172.21.15.32:0/4006564674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4330057bb0 con 0x7f4328004850 2023-11-24T07:58:38.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.655+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f432c06aed0 msgr2=0x7f432c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.655+0000 7f4345b70700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f432c06aed0 0x7f432c06d390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4334005040 tx=0x7f4334009030 comp rx=0 tx=0).stop 2023-11-24T07:58:38.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.655+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 msgr2=0x7f43401535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.655+0000 7f4345b70700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f43401535c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4330007340 tx=0x7f4330007650 comp rx=0 tx=0).stop 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.661+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 shutdown_connections 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.661+0000 7f4345b70700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f432c06aed0 0x7f432c06d390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.661+0000 7f4345b70700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43400b4cf0 0x7f43400b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.661+0000 7f4345b70700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4328004850 0x7f43401535c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.661+0000 7f4345b70700 1 --2- 172.21.15.32:0/4006564674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43400b3c10 0x7f43400ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.661+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 >> 172.21.15.32:0/4006564674 conn(0x7f434006b410 msgr2=0x7f434015e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.663+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 shutdown_connections 2023-11-24T07:58:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.663+0000 7f4345b70700 1 -- 172.21.15.32:0/4006564674 wait complete. 2023-11-24T07:58:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T07:58:38.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.814+0000 7f74e62fd700 1 Processor -- start 2023-11-24T07:58:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e62fd700 1 -- start start 2023-11-24T07:58:38.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e62fd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e00adf00 0x7f74e006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e62fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74e006c540 0x7f74e00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e62fd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e00af950 con 0x7f74e01446a0 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e62fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e0144a80 con 0x7f74e006c540 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e62fd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e006c9b0 con 0x7f74e00adf00 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.815+0000 7f74e489a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74e006c540 0x7f74e00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74e006c540 0x7f74e00b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56338/0 (socket says 172.21.15.32:56338) 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 -- 172.21.15.32:0/892643630 learned_addr learned my addr 172.21.15.32:0/892643630 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74df7fe700 1 --2- 172.21.15.32:0/892643630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e00adf00 0x7f74e006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 -- 172.21.15.32:0/892643630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e00adf00 msgr2=0x7f74e006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 --2- 172.21.15.32:0/892643630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e00adf00 0x7f74e006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 -- 172.21.15.32:0/892643630 >> v1:172.21.15.123:6789/0 conn(0x7f74e01446a0 legacy=0x7f74e00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 -- 172.21.15.32:0/892643630 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00afc30 con 0x7f74e006c540 2023-11-24T07:58:38.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74e489a700 1 --2- 172.21.15.32:0/892643630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e006c540 0x7f74e00b3d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009e80 tx=0x7f74d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801036a350a78305 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74deffd700 1 -- 172.21.15.32:0/892643630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800e070 con 0x7f74e006c540 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74deffd700 1 -- 172.21.15.32:0/892643630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800b980 con 0x7f74e006c540 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.816+0000 7f74deffd700 1 -- 172.21.15.32:0/892643630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800bd30 con 0x7f74e006c540 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 -- 172.21.15.32:0/892643630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e006c540 msgr2=0x7f74e00b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 --2- 172.21.15.32:0/892643630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e006c540 0x7f74e00b3d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009e80 tx=0x7f74d800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 -- 172.21.15.32:0/892643630 shutdown_connections 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 --2- 172.21.15.32:0/892643630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e006c540 0x7f74e00b3d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 --2- 172.21.15.32:0/892643630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e00adf00 0x7f74e006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 -- 172.21.15.32:0/892643630 >> 172.21.15.32:0/892643630 conn(0x7f74e0071b90 msgr2=0x7f74e0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 -- 172.21.15.32:0/892643630 shutdown_connections 2023-11-24T07:58:38.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.817+0000 7f74e62fd700 1 -- 172.21.15.32:0/892643630 wait complete. 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.818+0000 7f74e62fd700 1 Processor -- start 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.818+0000 7f74e62fd700 1 -- start start 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e62fd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e006c540 0x7f74e013a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e62fd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e00adf00 0x7f74e00cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e62fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 0x7f74e0139950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e62fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0138c60 con 0x7f74e01446a0 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e62fd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e013a900 con 0x7f74e00adf00 2023-11-24T07:58:38.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e62fd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0138e70 con 0x7f74e006c540 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 0x7f74e0139950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 0x7f74e0139950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56344/0 (socket says 172.21.15.32:56344) 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 -- 172.21.15.32:0/1827607654 learned_addr learned my addr 172.21.15.32:0/1827607654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74df7fe700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e00adf00 0x7f74e00cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74dffff700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e006c540 0x7f74e013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 -- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e006c540 msgr2=0x7f74e013a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e006c540 0x7f74e013a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 -- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e00adf00 msgr2=0x7f74e00cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e00adf00 0x7f74e00cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 -- 172.21.15.32:0/1827607654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d8009b30 con 0x7f74e01446a0 2023-11-24T07:58:38.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.819+0000 7f74e489a700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 0x7f74e0139950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74e00729d0 tx=0x7f74d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.820+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004800 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.820+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8004960 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.820+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e0139150 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.820+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d801c680 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.820+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e00d0450 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.821+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e00753f0 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.821+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74d801c7e0 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.821+0000 7f74dcff9700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f74cc06aed0 0x7f74cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.821+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74d808ad50 con 0x7f74e01446a0 2023-11-24T07:58:38.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.822+0000 7f74dffff700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f74cc06aed0 0x7f74cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.822+0000 7f74dffff700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f74cc06aed0 0x7f74cc06d390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74e015a330 tx=0x7f74d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:38.826+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74d808e030 con 0x7f74e01446a0 2023-11-24T07:58:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.028+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74e0067f60 con 0x7f74e01446a0 2023-11-24T07:58:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.029+0000 7f74dcff9700 1 -- 172.21.15.32:0/1827607654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16506 (secure 0 0 0) 0x7f74d8055e90 con 0x7f74e01446a0 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:38.060485+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f74cc06aed0 msgr2=0x7f74cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f74cc06aed0 0x7f74cc06d390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74e015a330 tx=0x7f74d0009040 comp rx=0 tx=0).stop 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 msgr2=0x7f74e0139950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 0x7f74e0139950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74e00729d0 tx=0x7f74d8004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 shutdown_connections 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f74cc06aed0 0x7f74cc06d390 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74e01446a0 0x7f74e0139950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74e006c540 0x7f74e013a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 --2- 172.21.15.32:0/1827607654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74e00adf00 0x7f74e00cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 >> 172.21.15.32:0/1827607654 conn(0x7f74e0071b90 msgr2=0x7f74e00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 shutdown_connections 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.031+0000 7f74e62fd700 1 -- 172.21.15.32:0/1827607654 wait complete. 2023-11-24T07:58:39.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.169+0000 7f37c1738700 1 Processor -- start 2023-11-24T07:58:39.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.169+0000 7f37c1738700 1 -- start start 2023-11-24T07:58:39.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37c1738700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37bc0adf00 0x7f37bc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37c1738700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc0af950 con 0x7f37bc1446a0 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37c1738700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc0afc30 con 0x7f37bc06c540 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37c1738700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc06c9f0 con 0x7f37bc0adf00 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37ba7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37bc0adf00 0x7f37bc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37ba7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37bc0adf00 0x7f37bc06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56350/0 (socket says 172.21.15.32:56350) 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37ba7fc700 1 -- 172.21.15.32:0/4136785909 learned_addr learned my addr 172.21.15.32:0/4136785909 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.170+0000 7f37baffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37bc1446a0 0x7f37bc0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58060/0 (socket says 172.21.15.32:58060) 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37ba7fc700 1 -- 172.21.15.32:0/4136785909 >> v1:172.21.15.123:6789/0 conn(0x7f37bc06c540 legacy=0x7f37bc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37ba7fc700 1 -- 172.21.15.32:0/4136785909 >> v1:172.21.15.32:6789/0 conn(0x7f37bc1446a0 legacy=0x7f37bc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37ba7fc700 1 -- 172.21.15.32:0/4136785909 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc0aff10 con 0x7f37bc0adf00 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37ba7fc700 1 --2- 172.21.15.32:0/4136785909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0adf00 0x7f37bc06c000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37a40098f0 tx=0x7f37a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874554731443988a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37b9ffb700 1 -- 172.21.15.32:0/4136785909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a400b8a0 con 0x7f37bc0adf00 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37b9ffb700 1 -- 172.21.15.32:0/4136785909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a400ba00 con 0x7f37bc0adf00 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.171+0000 7f37b9ffb700 1 -- 172.21.15.32:0/4136785909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a400bd90 con 0x7f37bc0adf00 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 -- 172.21.15.32:0/4136785909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0adf00 msgr2=0x7f37bc06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 --2- 172.21.15.32:0/4136785909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0adf00 0x7f37bc06c000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37a40098f0 tx=0x7f37a4009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 -- 172.21.15.32:0/4136785909 shutdown_connections 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 --2- 172.21.15.32:0/4136785909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc0adf00 0x7f37bc06c000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 -- 172.21.15.32:0/4136785909 >> 172.21.15.32:0/4136785909 conn(0x7f37bc071b90 msgr2=0x7f37bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 -- 172.21.15.32:0/4136785909 shutdown_connections 2023-11-24T07:58:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.172+0000 7f37c1738700 1 -- 172.21.15.32:0/4136785909 wait complete. 2023-11-24T07:58:39.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.173+0000 7f37c1738700 1 Processor -- start 2023-11-24T07:58:39.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.173+0000 7f37c1738700 1 -- start start 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37c1738700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 0x7f37bc1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37c1738700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc0adf00 0x7f37bc1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37c1738700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37bc1446a0 0x7f37bc1e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37c1738700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc138d50 con 0x7f37bc06c540 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37c1738700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc139030 con 0x7f37bc0adf00 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37c1738700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1421b0 con 0x7f37bc1446a0 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 0x7f37bc1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 0x7f37bc1f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56354/0 (socket says 172.21.15.32:56354) 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 -- 172.21.15.32:0/617913878 learned_addr learned my addr 172.21.15.32:0/617913878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37ba7fc700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc0adf00 0x7f37bc1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37bb7fe700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37bc1446a0 0x7f37bc1e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 -- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37bc1446a0 msgr2=0x7f37bc1e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37bc1446a0 0x7f37bc1e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 -- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc0adf00 msgr2=0x7f37bc1f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc0adf00 0x7f37bc1f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 -- 172.21.15.32:0/617913878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009550 con 0x7f37bc06c540 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37ba7fc700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc0adf00 0x7f37bc1f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37bb7fe700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37bc1446a0 0x7f37bc1e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37baffd700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 0x7f37bc1f5590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00c500 tx=0x7f37ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.174+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac005cb0 con 0x7f37bc06c540 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.175+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac005e10 con 0x7f37bc06c540 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.175+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc142490 con 0x7f37bc06c540 2023-11-24T07:58:39.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.175+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0072d0 con 0x7f37bc06c540 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.175+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc139310 con 0x7f37bc06c540 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.175+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc0753f0 con 0x7f37bc06c540 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.176+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37ac007430 con 0x7f37bc06c540 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.176+0000 7f37b37fe700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f379c06aed0 0x7f379c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.176+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37ac0889b0 con 0x7f37bc06c540 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.177+0000 7f37ba7fc700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f379c06aed0 0x7f379c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.177+0000 7f37ba7fc700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f379c06aed0 0x7f379c06d390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f37a40098f0 tx=0x7f37a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.181+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37ac053a70 con 0x7f37bc06c540 2023-11-24T07:58:39.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.383+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37bc067f60 con 0x7f37bc06c540 2023-11-24T07:58:39.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.384+0000 7f37b37fe700 1 -- 172.21.15.32:0/617913878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16506 (secure 0 0 0) 0x7f37ac017020 con 0x7f37bc06c540 2023-11-24T07:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:38.060485+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.386+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f379c06aed0 msgr2=0x7f379c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.386+0000 7f37c1738700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f379c06aed0 0x7f379c06d390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f37a40098f0 tx=0x7f37a4008030 comp rx=0 tx=0).stop 2023-11-24T07:58:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.386+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 msgr2=0x7f37bc1f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.386+0000 7f37c1738700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 0x7f37bc1f5590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00c500 tx=0x7f37ac00c8c0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 shutdown_connections 2023-11-24T07:58:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f379c06aed0 0x7f379c06d390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37bc1446a0 0x7f37bc1e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37bc06c540 0x7f37bc1f5590 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 --2- 172.21.15.32:0/617913878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc0adf00 0x7f37bc1f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 >> 172.21.15.32:0/617913878 conn(0x7f37bc071b90 msgr2=0x7f37bc0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 shutdown_connections 2023-11-24T07:58:39.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.387+0000 7f37c1738700 1 -- 172.21.15.32:0/617913878 wait complete. 2023-11-24T07:58:39.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.526+0000 7fab25cfc700 1 Processor -- start 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.526+0000 7fab25cfc700 1 -- start start 2023-11-24T07:58:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab25cfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab200add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab25cfc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab20073f90 0x7fab20074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab25cfc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2006b080 con 0x7fab2015f9e0 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab25cfc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab20074960 con 0x7fab20073f90 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab25cfc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2015c290 con 0x7fab200cf610 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab1ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab20073f90 0x7fab20074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab1effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab2015f9e0 0x7fab200b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58074/0 (socket says 172.21.15.32:58074) 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.527+0000 7fab1effd700 1 -- 172.21.15.32:0/952159042 learned_addr learned my addr 172.21.15.32:0/952159042 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1f7fe700 1 --2- 172.21.15.32:0/952159042 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab200add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1ffff700 1 -- 172.21.15.32:0/952159042 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 msgr2=0x7fab200add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1ffff700 1 --2- 172.21.15.32:0/952159042 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab200add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1ffff700 1 -- 172.21.15.32:0/952159042 >> v1:172.21.15.32:6789/0 conn(0x7fab2015f9e0 legacy=0x7fab200b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1ffff700 1 -- 172.21.15.32:0/952159042 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab200cf9f0 con 0x7fab20073f90 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1e7fc700 1 -- 172.21.15.32:0/952159042 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778192271 0 0) 0x7fab2006b080 con 0x7fab2015f9e0 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1ffff700 1 --2- 172.21.15.32:0/952159042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab20074420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fab18009d90 tx=0x7fab1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ab6fa27fafb775 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1e7fc700 1 -- 172.21.15.32:0/952159042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800e070 con 0x7fab20073f90 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1e7fc700 1 -- 172.21.15.32:0/952159042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1800bbd0 con 0x7fab20073f90 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.528+0000 7fab1e7fc700 1 -- 172.21.15.32:0/952159042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18012430 con 0x7fab20073f90 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 -- 172.21.15.32:0/952159042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 msgr2=0x7fab20074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 --2- 172.21.15.32:0/952159042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab20074420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fab18009d90 tx=0x7fab1800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 -- 172.21.15.32:0/952159042 shutdown_connections 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 --2- 172.21.15.32:0/952159042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab20074420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 --2- 172.21.15.32:0/952159042 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab200add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 -- 172.21.15.32:0/952159042 >> 172.21.15.32:0/952159042 conn(0x7fab200d0bf0 msgr2=0x7fab200d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 -- 172.21.15.32:0/952159042 shutdown_connections 2023-11-24T07:58:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.529+0000 7fab25cfc700 1 -- 172.21.15.32:0/952159042 wait complete. 2023-11-24T07:58:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.530+0000 7fab25cfc700 1 Processor -- start 2023-11-24T07:58:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.530+0000 7fab25cfc700 1 -- start start 2023-11-24T07:58:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab2006df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab2014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2015f9e0 0x7fab2015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2014de60 con 0x7fab20073f90 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2015c9a0 con 0x7fab2015f9e0 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab20153460 con 0x7fab200cf610 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab2006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab2006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56368/0 (socket says 172.21.15.32:56368) 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 -- 172.21.15.32:0/2052390617 learned_addr learned my addr 172.21.15.32:0/2052390617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1ffff700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2015f9e0 0x7fab2015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1effd700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab2014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 -- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 msgr2=0x7fab2014d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab2014d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 -- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2015f9e0 msgr2=0x7fab2015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2015f9e0 0x7fab2015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 -- 172.21.15.32:0/2052390617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009a70 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1ffff700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2015f9e0 0x7fab2015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab1f7fe700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab2006df90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e810 tx=0x7fab1000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10013610 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab10013770 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.531+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab20153740 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.532+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10013b20 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.532+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab20153a20 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.532+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab20068fb0 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.533+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab10013c80 con 0x7fab20073f90 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.533+0000 7fab17fff700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006aed0 0x7fab0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.533+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab10089280 con 0x7fab20073f90 2023-11-24T07:58:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.534+0000 7fab1effd700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006aed0 0x7fab0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.534+0000 7fab1effd700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006aed0 0x7fab0006d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab2006df20 tx=0x7fab08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.538+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab10054340 con 0x7fab20073f90 2023-11-24T07:58:39.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:39.790+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fab20160240 con 0x7fab20073f90 2023-11-24T07:58:40.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.657+0000 7fab17fff700 1 -- 172.21.15.32:0/2052390617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fab20160240 con 0x7fab20073f90 2023-11-24T07:58:40.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006aed0 msgr2=0x7fab0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006aed0 0x7fab0006d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab2006df20 tx=0x7fab08009040 comp rx=0 tx=0).stop 2023-11-24T07:58:40.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 msgr2=0x7fab2006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab2006df90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e810 tx=0x7fab1000ebd0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 shutdown_connections 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006aed0 0x7fab0006d390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab2015f9e0 0x7fab2015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab20073f90 0x7fab2006df90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 --2- 172.21.15.32:0/2052390617 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab200cf610 0x7fab2014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 >> 172.21.15.32:0/2052390617 conn(0x7fab200d0bf0 msgr2=0x7fab200b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 shutdown_connections 2023-11-24T07:58:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.660+0000 7fab25cfc700 1 -- 172.21.15.32:0/2052390617 wait complete. 2023-11-24T07:58:40.677 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:58:40.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:58:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.810+0000 7f5960574700 1 Processor -- start 2023-11-24T07:58:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.810+0000 7f5960574700 1 -- start start 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f5960574700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59581587f0 0x7f595806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f5960574700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59580b4370 0x7f595815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f5960574700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595806b530 con 0x7f595815b370 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f5960574700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580ae5e0 con 0x7f59581587f0 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f5960574700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595815be30 con 0x7f59580b4370 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59581587f0 0x7f595806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59581587f0 0x7f595806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56376/0 (socket says 172.21.15.32:56376) 2023-11-24T07:58:40.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 -- 172.21.15.32:0/3658130411 learned_addr learned my addr 172.21.15.32:0/3658130411 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595eb11700 1 --2- 172.21.15.32:0/3658130411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59580b4370 0x7f595815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 -- 172.21.15.32:0/3658130411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59580b4370 msgr2=0x7f595815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 --2- 172.21.15.32:0/3658130411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59580b4370 0x7f595815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 -- 172.21.15.32:0/3658130411 >> v1:172.21.15.32:6789/0 conn(0x7f595815b370 legacy=0x7f59580b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.811+0000 7f595e310700 1 -- 172.21.15.32:0/3658130411 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595815c110 con 0x7f59581587f0 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f595e310700 1 --2- 172.21.15.32:0/3658130411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59581587f0 0x7f595806aff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59480098f0 tx=0x7f5948009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54046b1c8c44c90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f595d30e700 1 -- 172.21.15.32:0/3658130411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948011830 con 0x7f59581587f0 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f595d30e700 1 -- 172.21.15.32:0/3658130411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948011990 con 0x7f59581587f0 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f595d30e700 1 -- 172.21.15.32:0/3658130411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948011d20 con 0x7f59581587f0 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 -- 172.21.15.32:0/3658130411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59581587f0 msgr2=0x7f595806aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 --2- 172.21.15.32:0/3658130411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59581587f0 0x7f595806aff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59480098f0 tx=0x7f5948009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 -- 172.21.15.32:0/3658130411 shutdown_connections 2023-11-24T07:58:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 --2- 172.21.15.32:0/3658130411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59580b4370 0x7f595815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 --2- 172.21.15.32:0/3658130411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59581587f0 0x7f595806aff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 -- 172.21.15.32:0/3658130411 >> 172.21.15.32:0/3658130411 conn(0x7f59580d0bf0 msgr2=0x7f59580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 -- 172.21.15.32:0/3658130411 shutdown_connections 2023-11-24T07:58:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.812+0000 7f5960574700 1 -- 172.21.15.32:0/3658130411 wait complete. 2023-11-24T07:58:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 Processor -- start 2023-11-24T07:58:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 -- start start 2023-11-24T07:58:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 0x7f59581f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59581587f0 0x7f595814d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f595815b370 0x7f5958159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59581f3ac0 con 0x7f59580b4370 2023-11-24T07:58:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59581518a0 con 0x7f59581587f0 2023-11-24T07:58:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f5960574700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59581f8050 con 0x7f595815b370 2023-11-24T07:58:40.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f595e310700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 0x7f59581f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f595e310700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 0x7f59581f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56382/0 (socket says 172.21.15.32:56382) 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.814+0000 7f595e310700 1 -- 172.21.15.32:0/3319724260 learned_addr learned my addr 172.21.15.32:0/3319724260 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595eb11700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f595815b370 0x7f5958159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595db0f700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59581587f0 0x7f595814d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595e310700 1 -- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f595815b370 msgr2=0x7f5958159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595e310700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f595815b370 0x7f5958159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595e310700 1 -- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59581587f0 msgr2=0x7f595814d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595e310700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59581587f0 0x7f595814d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595e310700 1 -- 172.21.15.32:0/3319724260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009550 con 0x7f59580b4370 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f595e310700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 0x7f59581f7b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5948009c40 tx=0x7f5948004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948004530 con 0x7f59580b4370 2023-11-24T07:58:40.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594801f070 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595814dc80 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480038c0 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.815+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581f3cb0 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.816+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958068fb0 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.816+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5948021030 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.817+0000 7f594effd700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f593c06aed0 0x7f593c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.817+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f594808aa10 con 0x7f59580b4370 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.817+0000 7f595db0f700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f593c06aed0 0x7f593c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.818+0000 7f595db0f700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f593c06aed0 0x7f593c06d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5954007fd0 tx=0x7f5954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:40.821+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5948055b50 con 0x7f59580b4370 2023-11-24T07:58:41.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.070+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5958155240 con 0x7f59580b4370 2023-11-24T07:58:41.073 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.662+0000 7f594effd700 1 -- 172.21.15.32:0/3319724260 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5948015020 con 0x7f59580b4370 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f593c06aed0 msgr2=0x7f593c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f593c06aed0 0x7f593c06d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5954007fd0 tx=0x7f5954009040 comp rx=0 tx=0).stop 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 msgr2=0x7f59581f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 0x7f59581f7b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5948009c40 tx=0x7f5948004750 comp rx=0 tx=0).stop 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 shutdown_connections 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f593c06aed0 0x7f593c06d390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f595815b370 0x7f5958159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59580b4370 0x7f59581f7b10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 --2- 172.21.15.32:0/3319724260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59581587f0 0x7f595814d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 >> 172.21.15.32:0/3319724260 conn(0x7f59580d0bf0 msgr2=0x7f595806ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 shutdown_connections 2023-11-24T07:58:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.665+0000 7f5960574700 1 -- 172.21.15.32:0/3319724260 wait complete. 2023-11-24T07:58:41.681 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:58:41.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.811+0000 7f965002e700 1 Processor -- start 2023-11-24T07:58:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.811+0000 7f965002e700 1 -- start start 2023-11-24T07:58:41.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f965002e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96481446a0 0x7f96480b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f965002e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964806c710 0x7f96480b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f965002e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96480af950 con 0x7f96480adf00 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f965002e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9648144a80 con 0x7f964806c710 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f965002e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96480afc30 con 0x7f96481446a0 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964e5cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964806c710 0x7f96480b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964e5cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964806c710 0x7f96480b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56394/0 (socket says 172.21.15.32:56394) 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964e5cb700 1 -- 172.21.15.32:0/2684351040 learned_addr learned my addr 172.21.15.32:0/2684351040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964ddca700 1 --2- 172.21.15.32:0/2684351040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96481446a0 0x7f96480b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964e5cb700 1 -- 172.21.15.32:0/2684351040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96481446a0 msgr2=0x7f96480b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:41.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964e5cb700 1 --2- 172.21.15.32:0/2684351040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96481446a0 0x7f96480b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.812+0000 7f964e5cb700 1 -- 172.21.15.32:0/2684351040 >> v1:172.21.15.123:6789/0 conn(0x7f96480adf00 legacy=0x7f964806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f964e5cb700 1 -- 172.21.15.32:0/2684351040 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96480aff10 con 0x7f964806c710 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f964e5cb700 1 --2- 172.21.15.32:0/2684351040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f96480b3d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9640009e50 tx=0x7f964000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138939aade2d821f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f964cdc8700 1 -- 172.21.15.32:0/2684351040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964000e070 con 0x7f964806c710 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f964cdc8700 1 -- 172.21.15.32:0/2684351040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964000b9c0 con 0x7f964806c710 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f964cdc8700 1 -- 172.21.15.32:0/2684351040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964000bd70 con 0x7f964806c710 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 -- 172.21.15.32:0/2684351040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 msgr2=0x7f96480b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 --2- 172.21.15.32:0/2684351040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f96480b3d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9640009e50 tx=0x7f964000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 -- 172.21.15.32:0/2684351040 shutdown_connections 2023-11-24T07:58:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 --2- 172.21.15.32:0/2684351040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f96480b3d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 --2- 172.21.15.32:0/2684351040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96481446a0 0x7f96480b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 -- 172.21.15.32:0/2684351040 >> 172.21.15.32:0/2684351040 conn(0x7f9648071b90 msgr2=0x7f9648071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 -- 172.21.15.32:0/2684351040 shutdown_connections 2023-11-24T07:58:41.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.813+0000 7f965002e700 1 -- 172.21.15.32:0/2684351040 wait complete. 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 Processor -- start 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 -- start start 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f964815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96480adf00 0x7f96480cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96481446a0 0x7f96480cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964815dd20 con 0x7f964806c710 2023-11-24T07:58:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964815a880 con 0x7f96481446a0 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.815+0000 7f965002e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964815d110 con 0x7f96480adf00 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f964815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f964815a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56396/0 (socket says 172.21.15.32:56396) 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 -- 172.21.15.32:0/2863759597 learned_addr learned my addr 172.21.15.32:0/2863759597 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964e5cb700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96481446a0 0x7f96480cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964d5c9700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96480adf00 0x7f96480cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 -- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96480adf00 msgr2=0x7f96480cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96480adf00 0x7f96480cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 -- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96481446a0 msgr2=0x7f96480cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96481446a0 0x7f96480cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 -- 172.21.15.32:0/2863759597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640009b30 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964e5cb700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96481446a0 0x7f96480cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f964ddca700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f964815a340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f963800d800 tx=0x7f963800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96380125f0 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638012750 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964815d3f0 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638012b00 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.816+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964815d9b0 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.817+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96480753f0 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.818+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9638012c60 con 0x7f964806c710 2023-11-24T07:58:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.818+0000 7f963e7fc700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f962c06aed0 0x7f962c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.818+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96380897c0 con 0x7f964806c710 2023-11-24T07:58:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.818+0000 7f964d5c9700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f962c06aed0 0x7f962c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.819+0000 7f964d5c9700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f962c06aed0 0x7f962c06d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96481415b0 tx=0x7f9644009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:41.822+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9638054870 con 0x7f964806c710 2023-11-24T07:58:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.070+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9648067f60 con 0x7f964806c710 2023-11-24T07:58:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.070+0000 7f963e7fc700 1 -- 172.21.15.32:0/2863759597 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f9638057e90 con 0x7f964806c710 2023-11-24T07:58:42.072 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:42.072 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f962c06aed0 msgr2=0x7f962c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f962c06aed0 0x7f962c06d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96481415b0 tx=0x7f9644009040 comp rx=0 tx=0).stop 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 msgr2=0x7f964815a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f964815a340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f963800d800 tx=0x7f963800db10 comp rx=0 tx=0).stop 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 shutdown_connections 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f962c06aed0 0x7f962c06d390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96481446a0 0x7f96480cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964806c710 0x7f964815a340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 --2- 172.21.15.32:0/2863759597 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96480adf00 0x7f96480cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 >> 172.21.15.32:0/2863759597 conn(0x7f9648071b90 msgr2=0x7f96480b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 shutdown_connections 2023-11-24T07:58:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.073+0000 7f965002e700 1 -- 172.21.15.32:0/2863759597 wait complete. 2023-11-24T07:58:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:58:42.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.210+0000 7f07dad6d700 1 Processor -- start 2023-11-24T07:58:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.210+0000 7f07dad6d700 1 -- start start 2023-11-24T07:58:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.210+0000 7f07dad6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d40d22f0 0x7f07d40d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.210+0000 7f07dad6d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d40d0150 con 0x7f07d40ce4d0 2023-11-24T07:58:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.210+0000 7f07dad6d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d40cee50 con 0x7f07d415e080 2023-11-24T07:58:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.210+0000 7f07dad6d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d415e460 con 0x7f07d40d22f0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07d40ce4d0 0x7f07d40d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58116/0 (socket says 172.21.15.32:58116) 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d3fff700 1 -- 172.21.15.32:0/2753943816 learned_addr learned my addr 172.21.15.32:0/2753943816 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d930a700 1 --2- 172.21.15.32:0/2753943816 >> v2:172.21.15.32:3300/0 conn(0x7f07d40d22f0 0x7f07d40d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d930a700 1 -- 172.21.15.32:0/2753943816 >> v1:172.21.15.123:6789/0 conn(0x7f07d415e080 legacy=0x7f07d40cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d930a700 1 -- 172.21.15.32:0/2753943816 >> v1:172.21.15.32:6789/0 conn(0x7f07d40ce4d0 legacy=0x7f07d40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d930a700 1 -- 172.21.15.32:0/2753943816 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40cf130 con 0x7f07d40d22f0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d37fe700 1 -- 172.21.15.32:0/2753943816 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24581357 0 0) 0x7f07d40d0150 con 0x7f07d40ce4d0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d930a700 1 --2- 172.21.15.32:0/2753943816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40d22f0 0x7f07d40d05e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009d90 tx=0x7f07cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7db253d897eec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d37fe700 1 -- 172.21.15.32:0/2753943816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00e070 con 0x7f07d40d22f0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d37fe700 1 -- 172.21.15.32:0/2753943816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00bb90 con 0x7f07d40d22f0 2023-11-24T07:58:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.211+0000 7f07d37fe700 1 -- 172.21.15.32:0/2753943816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc003a20 con 0x7f07d40d22f0 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 -- 172.21.15.32:0/2753943816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40d22f0 msgr2=0x7f07d40d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 --2- 172.21.15.32:0/2753943816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40d22f0 0x7f07d40d05e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009d90 tx=0x7f07cc00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 -- 172.21.15.32:0/2753943816 shutdown_connections 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 --2- 172.21.15.32:0/2753943816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40d22f0 0x7f07d40d05e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 -- 172.21.15.32:0/2753943816 >> 172.21.15.32:0/2753943816 conn(0x7f07d4073130 msgr2=0x7f07d409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 -- 172.21.15.32:0/2753943816 shutdown_connections 2023-11-24T07:58:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.212+0000 7f07dad6d700 1 -- 172.21.15.32:0/2753943816 wait complete. 2023-11-24T07:58:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.213+0000 7f07dad6d700 1 Processor -- start 2023-11-24T07:58:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.213+0000 7f07dad6d700 1 -- start start 2023-11-24T07:58:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07dad6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 0x7f07d415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07dad6d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07d40d22f0 0x7f07d40b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07dad6d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d415e080 0x7f07d40b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07dad6d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d41f3c90 con 0x7f07d40ce4d0 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07dad6d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d41f3f70 con 0x7f07d415e080 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07dad6d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d41f4250 con 0x7f07d40d22f0 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 0x7f07d415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 0x7f07d415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56410/0 (socket says 172.21.15.32:56410) 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 -- 172.21.15.32:0/276774860 learned_addr learned my addr 172.21.15.32:0/276774860 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d3fff700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07d40d22f0 0x7f07d40b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d930a700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d415e080 0x7f07d40b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 -- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07d40d22f0 msgr2=0x7f07d40b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07d40d22f0 0x7f07d40b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 -- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d415e080 msgr2=0x7f07d40b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d415e080 0x7f07d40b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 -- 172.21.15.32:0/276774860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc009a70 con 0x7f07d40ce4d0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.214+0000 7f07d8b09700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 0x7f07d415cb20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07c800b4a0 tx=0x7f07c800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.215+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8011830 con 0x7f07d40ce4d0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.215+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c8011990 con 0x7f07d40ce4d0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.215+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d41f4530 con 0x7f07d40ce4d0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.215+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8011d40 con 0x7f07d40ce4d0 2023-11-24T07:58:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.215+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d41f7f60 con 0x7f07d40ce4d0 2023-11-24T07:58:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.216+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d4067f60 con 0x7f07d40ce4d0 2023-11-24T07:58:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.216+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07c801a400 con 0x7f07d40ce4d0 2023-11-24T07:58:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.216+0000 7f07d17fa700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f07b806aed0 0x7f07b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.216+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07c80892f0 con 0x7f07d40ce4d0 2023-11-24T07:58:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.217+0000 7f07d3fff700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f07b806aed0 0x7f07b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.217+0000 7f07d3fff700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f07b806aed0 0x7f07b806d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07d40c8660 tx=0x7f07c0007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.221+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07c8054430 con 0x7f07d40ce4d0 2023-11-24T07:58:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.423+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07d40d19c0 con 0x7f07d40ce4d0 2023-11-24T07:58:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.424+0000 7f07d17fa700 1 -- 172.21.15.32:0/276774860 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f07c8057a50 con 0x7f07d40ce4d0 2023-11-24T07:58:42.426 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:42.426 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.426+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f07b806aed0 msgr2=0x7f07b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.426+0000 7f07dad6d700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f07b806aed0 0x7f07b806d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07d40c8660 tx=0x7f07c0007be0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.426+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 msgr2=0x7f07d415cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.426+0000 7f07dad6d700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 0x7f07d415cb20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07c800b4a0 tx=0x7f07c800b860 comp rx=0 tx=0).stop 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 shutdown_connections 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f07b806aed0 0x7f07b806d390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d415e080 0x7f07d40b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d40ce4d0 0x7f07d415cb20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 --2- 172.21.15.32:0/276774860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07d40d22f0 0x7f07d40b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 >> 172.21.15.32:0/276774860 conn(0x7f07d4073130 msgr2=0x7f07d406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 shutdown_connections 2023-11-24T07:58:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.427+0000 7f07dad6d700 1 -- 172.21.15.32:0/276774860 wait complete. 2023-11-24T07:58:42.441 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:58:42.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.565+0000 7f1ddf51f700 1 Processor -- start 2023-11-24T07:58:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.565+0000 7f1ddf51f700 1 -- start start 2023-11-24T07:58:42.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddf51f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1dd815bef0 0x7f1dd815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddf51f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd806dd80 con 0x7f1dd81624d0 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddf51f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd815f880 con 0x7f1dd80afe70 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddf51f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd815ed90 con 0x7f1dd815bef0 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddd2bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1dd815bef0 0x7f1dd815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1dddabc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1dd81624d0 0x7f1dd806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58134/0 (socket says 172.21.15.32:58134) 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddd2bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1dd815bef0 0x7f1dd815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56418/0 (socket says 172.21.15.32:56418) 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1dddabc700 1 -- 172.21.15.32:0/3197423783 learned_addr learned my addr 172.21.15.32:0/3197423783 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddd2bb700 1 -- 172.21.15.32:0/3197423783 >> v1:172.21.15.123:6789/0 conn(0x7f1dd80afe70 legacy=0x7f1dd80b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1ddd2bb700 1 -- 172.21.15.32:0/3197423783 >> v1:172.21.15.32:6789/0 conn(0x7f1dd81624d0 legacy=0x7f1dd806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddd2bb700 1 -- 172.21.15.32:0/3197423783 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd815c2d0 con 0x7f1dd815bef0 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.566+0000 7f1dcbfff700 1 -- 172.21.15.32:0/3197423783 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878583741 0 0) 0x7f1dd806dd80 con 0x7f1dd81624d0 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddd2bb700 1 --2- 172.21.15.32:0/3197423783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd815bef0 0x7f1dd815e850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009840 tx=0x7f1dd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb1ee22564c384a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1dcbfff700 1 -- 172.21.15.32:0/3197423783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4011830 con 0x7f1dd815bef0 2023-11-24T07:58:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1dcbfff700 1 -- 172.21.15.32:0/3197423783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4011990 con 0x7f1dd815bef0 2023-11-24T07:58:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1dcbfff700 1 -- 172.21.15.32:0/3197423783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4011d20 con 0x7f1dd815bef0 2023-11-24T07:58:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3197423783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd815bef0 msgr2=0x7f1dd815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3197423783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd815bef0 0x7f1dd815e850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009840 tx=0x7f1dd4009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3197423783 shutdown_connections 2023-11-24T07:58:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3197423783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd815bef0 0x7f1dd815e850 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3197423783 >> 172.21.15.32:0/3197423783 conn(0x7f1dd8071ac0 msgr2=0x7f1dd8071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.567+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3197423783 shutdown_connections 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.568+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3197423783 wait complete. 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 Processor -- start 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 -- start start 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd80afe70 0x7f1dd81578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dd815bef0 0x7f1dd815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 0x7f1dd815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd815ad70 con 0x7f1dd81624d0 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8157e10 con 0x7f1dd80afe70 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.569+0000 7f1ddf51f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd815a160 con 0x7f1dd815bef0 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 0x7f1dd815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 0x7f1dd815db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56428/0 (socket says 172.21.15.32:56428) 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 -- 172.21.15.32:0/3705173190 learned_addr learned my addr 172.21.15.32:0/3705173190 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1ddd2bb700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd80afe70 0x7f1dd81578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1ddcaba700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dd815bef0 0x7f1dd815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 -- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dd815bef0 msgr2=0x7f1dd815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dd815bef0 0x7f1dd815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 -- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd80afe70 msgr2=0x7f1dd81578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd80afe70 0x7f1dd81578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 -- 172.21.15.32:0/3705173190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40094a0 con 0x7f1dd81624d0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dddabc700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 0x7f1dd815db50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0002230 tx=0x7f1dd000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd000aac0 con 0x7f1dd81624d0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd000ac40 con 0x7f1dd81624d0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd815a440 con 0x7f1dd81624d0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00070d0 con 0x7f1dd81624d0 2023-11-24T07:58:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.570+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd815aa00 con 0x7f1dd81624d0 2023-11-24T07:58:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.571+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd815aee0 con 0x7f1dd81624d0 2023-11-24T07:58:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.572+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1dd0007250 con 0x7f1dd81624d0 2023-11-24T07:58:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.572+0000 7f1dc9ffb700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dbc06aed0 0x7f1dbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.572+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dd00893c0 con 0x7f1dd81624d0 2023-11-24T07:58:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.573+0000 7f1ddd2bb700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dbc06aed0 0x7f1dbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.573+0000 7f1ddd2bb700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dbc06aed0 0x7f1dbc06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40090b0 tx=0x7f1dd401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.577+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1dd0054470 con 0x7f1dd81624d0 2023-11-24T07:58:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.774+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1dd815f5a0 con 0x7f1dbc06aed0 2023-11-24T07:58:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.782+0000 7f1dc9ffb700 1 -- 172.21.15.32:0/3705173190 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264658 (secure 0 0 0) 0x7f1dd4011c00 con 0x7f1dbc06aed0 2023-11-24T07:58:42.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dbc06aed0 msgr2=0x7f1dbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dbc06aed0 0x7f1dbc06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40090b0 tx=0x7f1dd401c040 comp rx=0 tx=0).stop 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 msgr2=0x7f1dd815db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 0x7f1dd815db50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0002230 tx=0x7f1dd000cf50 comp rx=0 tx=0).stop 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 shutdown_connections 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dbc06aed0 0x7f1dbc06d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dd81624d0 0x7f1dd815db50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd80afe70 0x7f1dd81578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 --2- 172.21.15.32:0/3705173190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dd815bef0 0x7f1dd815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 >> 172.21.15.32:0/3705173190 conn(0x7f1dd8071ac0 msgr2=0x7f1dd80b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.785+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 shutdown_connections 2023-11-24T07:58:42.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.786+0000 7f1ddf51f700 1 -- 172.21.15.32:0/3705173190 wait complete. 2023-11-24T07:58:42.790 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T07:58:42.799 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-24T07:58:42.530316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219104,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755616,"statfs":{"total":773094113280,"available":772869750784,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.896835"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210697+0000","last_change":"2023-11-24T07:58:38.673717+0000","last_active":"2023-11-24T07:58:40.210697+0000","last_peered":"2023-11-24T07:58:40.210697+0000","last_clean":"2023-11-24T07:58:40.210697+0000","last_became_active":"2023-11-24T07:58:38.673289+0000","last_became_peered":"2023-11-24T07:58:38.673289+0000","last_unstale":"2023-11-24T07:58:40.210697+0000","last_undegraded":"2023-11-24T07:58:40.210697+0000","last_fullsized":"2023-11-24T07:58:40.210697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:16:17.544102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209942+0000","last_change":"2023-11-24T07:58:38.666572+0000","last_active":"2023-11-24T07:58:40.209942+0000","last_peered":"2023-11-24T07:58:40.209942+0000","last_clean":"2023-11-24T07:58:40.209942+0000","last_became_active":"2023-11-24T07:58:38.666220+0000","last_became_peered":"2023-11-24T07:58:38.666220+0000","last_unstale":"2023-11-24T07:58:40.209942+0000","last_undegraded":"2023-11-24T07:58:40.209942+0000","last_fullsized":"2023-11-24T07:58:40.209942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670074+0000","last_change":"2023-11-24T07:58:38.664139+0000","last_active":"2023-11-24T07:58:39.670074+0000","last_peered":"2023-11-24T07:58:39.670074+0000","last_clean":"2023-11-24T07:58:39.670074+0000","last_became_active":"2023-11-24T07:58:38.660601+0000","last_became_peered":"2023-11-24T07:58:38.660601+0000","last_unstale":"2023-11-24T07:58:39.670074+0000","last_undegraded":"2023-11-24T07:58:39.670074+0000","last_fullsized":"2023-11-24T07:58:39.670074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662796+0000","last_change":"2023-11-24T07:58:38.670834+0000","last_active":"2023-11-24T07:58:39.662796+0000","last_peered":"2023-11-24T07:58:39.662796+0000","last_clean":"2023-11-24T07:58:39.662796+0000","last_became_active":"2023-11-24T07:58:38.670567+0000","last_became_peered":"2023-11-24T07:58:38.670567+0000","last_unstale":"2023-11-24T07:58:39.662796+0000","last_undegraded":"2023-11-24T07:58:39.662796+0000","last_fullsized":"2023-11-24T07:58:39.662796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210112+0000","last_change":"2023-11-24T07:58:38.657900+0000","last_active":"2023-11-24T07:58:40.210112+0000","last_peered":"2023-11-24T07:58:40.210112+0000","last_clean":"2023-11-24T07:58:40.210112+0000","last_became_active":"2023-11-24T07:58:38.657374+0000","last_became_peered":"2023-11-24T07:58:38.657374+0000","last_unstale":"2023-11-24T07:58:40.210112+0000","last_undegraded":"2023-11-24T07:58:40.210112+0000","last_fullsized":"2023-11-24T07:58:40.210112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662085+0000","last_change":"2023-11-24T07:58:38.668973+0000","last_active":"2023-11-24T07:58:39.662085+0000","last_peered":"2023-11-24T07:58:39.662085+0000","last_clean":"2023-11-24T07:58:39.662085+0000","last_became_active":"2023-11-24T07:58:38.668660+0000","last_became_peered":"2023-11-24T07:58:38.668660+0000","last_unstale":"2023-11-24T07:58:39.662085+0000","last_undegraded":"2023-11-24T07:58:39.662085+0000","last_fullsized":"2023-11-24T07:58:39.662085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209727+0000","last_change":"2023-11-24T07:58:38.670604+0000","last_active":"2023-11-24T07:58:40.209727+0000","last_peered":"2023-11-24T07:58:40.209727+0000","last_clean":"2023-11-24T07:58:40.209727+0000","last_became_active":"2023-11-24T07:58:38.670304+0000","last_became_peered":"2023-11-24T07:58:38.670304+0000","last_unstale":"2023-11-24T07:58:40.209727+0000","last_undegraded":"2023-11-24T07:58:40.209727+0000","last_fullsized":"2023-11-24T07:58:40.209727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655626+0000","last_change":"2023-11-24T07:58:38.671094+0000","last_active":"2023-11-24T07:58:39.655626+0000","last_peered":"2023-11-24T07:58:39.655626+0000","last_clean":"2023-11-24T07:58:39.655626+0000","last_became_active":"2023-11-24T07:58:38.670534+0000","last_became_peered":"2023-11-24T07:58:38.670534+0000","last_unstale":"2023-11-24T07:58:39.655626+0000","last_undegraded":"2023-11-24T07:58:39.655626+0000","last_fullsized":"2023-11-24T07:58:39.655626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210625+0000","last_change":"2023-11-24T07:58:38.673877+0000","last_active":"2023-11-24T07:58:40.210625+0000","last_peered":"2023-11-24T07:58:40.210625+0000","last_clean":"2023-11-24T07:58:40.210625+0000","last_became_active":"2023-11-24T07:58:38.673449+0000","last_became_peered":"2023-11-24T07:58:38.673449+0000","last_unstale":"2023-11-24T07:58:40.210625+0000","last_undegraded":"2023-11-24T07:58:40.210625+0000","last_fullsized":"2023-11-24T07:58:40.210625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661371+0000","last_change":"2023-11-24T07:58:38.667090+0000","last_active":"2023-11-24T07:58:39.661371+0000","last_peered":"2023-11-24T07:58:39.661371+0000","last_clean":"2023-11-24T07:58:39.661371+0000","last_became_active":"2023-11-24T07:58:38.665486+0000","last_became_peered":"2023-11-24T07:58:38.665486+0000","last_unstale":"2023-11-24T07:58:39.661371+0000","last_undegraded":"2023-11-24T07:58:39.661371+0000","last_fullsized":"2023-11-24T07:58:39.661371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661214+0000","last_change":"2023-11-24T07:58:38.666051+0000","last_active":"2023-11-24T07:58:39.661214+0000","last_peered":"2023-11-24T07:58:39.661214+0000","last_clean":"2023-11-24T07:58:39.661214+0000","last_became_active":"2023-11-24T07:58:38.665574+0000","last_became_peered":"2023-11-24T07:58:38.665574+0000","last_unstale":"2023-11-24T07:58:39.661214+0000","last_undegraded":"2023-11-24T07:58:39.661214+0000","last_fullsized":"2023-11-24T07:58:39.661214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210541+0000","last_change":"2023-11-24T07:58:38.667776+0000","last_active":"2023-11-24T07:58:40.210541+0000","last_peered":"2023-11-24T07:58:40.210541+0000","last_clean":"2023-11-24T07:58:40.210541+0000","last_became_active":"2023-11-24T07:58:38.667513+0000","last_became_peered":"2023-11-24T07:58:38.667513+0000","last_unstale":"2023-11-24T07:58:40.210541+0000","last_undegraded":"2023-11-24T07:58:40.210541+0000","last_fullsized":"2023-11-24T07:58:40.210541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655517+0000","last_change":"2023-11-24T07:58:38.672201+0000","last_active":"2023-11-24T07:58:39.655517+0000","last_peered":"2023-11-24T07:58:39.655517+0000","last_clean":"2023-11-24T07:58:39.655517+0000","last_became_active":"2023-11-24T07:58:38.671869+0000","last_became_peered":"2023-11-24T07:58:38.671869+0000","last_unstale":"2023-11-24T07:58:39.655517+0000","last_undegraded":"2023-11-24T07:58:39.655517+0000","last_fullsized":"2023-11-24T07:58:39.655517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670156+0000","last_change":"2023-11-24T07:58:38.662685+0000","last_active":"2023-11-24T07:58:39.670156+0000","last_peered":"2023-11-24T07:58:39.670156+0000","last_clean":"2023-11-24T07:58:39.670156+0000","last_became_active":"2023-11-24T07:58:38.660043+0000","last_became_peered":"2023-11-24T07:58:38.660043+0000","last_unstale":"2023-11-24T07:58:39.670156+0000","last_undegraded":"2023-11-24T07:58:39.670156+0000","last_fullsized":"2023-11-24T07:58:39.670156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209459+0000","last_change":"2023-11-24T07:58:38.667653+0000","last_active":"2023-11-24T07:58:40.209459+0000","last_peered":"2023-11-24T07:58:40.209459+0000","last_clean":"2023-11-24T07:58:40.209459+0000","last_became_active":"2023-11-24T07:58:38.667388+0000","last_became_peered":"2023-11-24T07:58:38.667388+0000","last_unstale":"2023-11-24T07:58:40.209459+0000","last_undegraded":"2023-11-24T07:58:40.209459+0000","last_fullsized":"2023-11-24T07:58:40.209459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661870+0000","last_change":"2023-11-24T07:58:38.669289+0000","last_active":"2023-11-24T07:58:39.661870+0000","last_peered":"2023-11-24T07:58:39.661870+0000","last_clean":"2023-11-24T07:58:39.661870+0000","last_became_active":"2023-11-24T07:58:38.668715+0000","last_became_peered":"2023-11-24T07:58:38.668715+0000","last_unstale":"2023-11-24T07:58:39.661870+0000","last_undegraded":"2023-11-24T07:58:39.661870+0000","last_fullsized":"2023-11-24T07:58:39.661870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662528+0000","last_change":"2023-11-24T07:58:38.674765+0000","last_active":"2023-11-24T07:58:39.662528+0000","last_peered":"2023-11-24T07:58:39.662528+0000","last_clean":"2023-11-24T07:58:39.662528+0000","last_became_active":"2023-11-24T07:58:38.674479+0000","last_became_peered":"2023-11-24T07:58:38.674479+0000","last_unstale":"2023-11-24T07:58:39.662528+0000","last_undegraded":"2023-11-24T07:58:39.662528+0000","last_fullsized":"2023-11-24T07:58:39.662528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210293+0000","last_change":"2023-11-24T07:58:38.667446+0000","last_active":"2023-11-24T07:58:40.210293+0000","last_peered":"2023-11-24T07:58:40.210293+0000","last_clean":"2023-11-24T07:58:40.210293+0000","last_became_active":"2023-11-24T07:58:38.667164+0000","last_became_peered":"2023-11-24T07:58:38.667164+0000","last_unstale":"2023-11-24T07:58:40.210293+0000","last_undegraded":"2023-11-24T07:58:40.210293+0000","last_fullsized":"2023-11-24T07:58:40.210293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210359+0000","last_change":"2023-11-24T07:58:38.663509+0000","last_active":"2023-11-24T07:58:40.210359+0000","last_peered":"2023-11-24T07:58:40.210359+0000","last_clean":"2023-11-24T07:58:40.210359+0000","last_became_active":"2023-11-24T07:58:38.661576+0000","last_became_peered":"2023-11-24T07:58:38.661576+0000","last_unstale":"2023-11-24T07:58:40.210359+0000","last_undegraded":"2023-11-24T07:58:40.210359+0000","last_fullsized":"2023-11-24T07:58:40.210359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669966+0000","last_change":"2023-11-24T07:58:37.664997+0000","last_active":"2023-11-24T07:58:39.669966+0000","last_peered":"2023-11-24T07:58:39.669966+0000","last_clean":"2023-11-24T07:58:39.669966+0000","last_became_active":"2023-11-24T07:58:37.664509+0000","last_became_peered":"2023-11-24T07:58:37.664509+0000","last_unstale":"2023-11-24T07:58:39.669966+0000","last_undegraded":"2023-11-24T07:58:39.669966+0000","last_fullsized":"2023-11-24T07:58:39.669966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656179+0000","last_change":"2023-11-24T07:58:38.671945+0000","last_active":"2023-11-24T07:58:39.656179+0000","last_peered":"2023-11-24T07:58:39.656179+0000","last_clean":"2023-11-24T07:58:39.656179+0000","last_became_active":"2023-11-24T07:58:38.671503+0000","last_became_peered":"2023-11-24T07:58:38.671503+0000","last_unstale":"2023-11-24T07:58:39.656179+0000","last_undegraded":"2023-11-24T07:58:39.656179+0000","last_fullsized":"2023-11-24T07:58:39.656179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656004+0000","last_change":"2023-11-24T07:58:37.664969+0000","last_active":"2023-11-24T07:58:39.656004+0000","last_peered":"2023-11-24T07:58:39.656004+0000","last_clean":"2023-11-24T07:58:39.656004+0000","last_became_active":"2023-11-24T07:58:37.664661+0000","last_became_peered":"2023-11-24T07:58:37.664661+0000","last_unstale":"2023-11-24T07:58:39.656004+0000","last_undegraded":"2023-11-24T07:58:39.656004+0000","last_fullsized":"2023-11-24T07:58:39.656004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670016+0000","last_change":"2023-11-24T07:58:38.670070+0000","last_active":"2023-11-24T07:58:39.670016+0000","last_peered":"2023-11-24T07:58:39.670016+0000","last_clean":"2023-11-24T07:58:39.670016+0000","last_became_active":"2023-11-24T07:58:38.669739+0000","last_became_peered":"2023-11-24T07:58:38.669739+0000","last_unstale":"2023-11-24T07:58:39.670016+0000","last_undegraded":"2023-11-24T07:58:39.670016+0000","last_fullsized":"2023-11-24T07:58:39.670016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:04.076457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210695+0000","last_change":"2023-11-24T07:58:37.663122+0000","last_active":"2023-11-24T07:58:40.210695+0000","last_peered":"2023-11-24T07:58:40.210695+0000","last_clean":"2023-11-24T07:58:40.210695+0000","last_became_active":"2023-11-24T07:58:37.662846+0000","last_became_peered":"2023-11-24T07:58:37.662846+0000","last_unstale":"2023-11-24T07:58:40.210695+0000","last_undegraded":"2023-11-24T07:58:40.210695+0000","last_fullsized":"2023-11-24T07:58:40.210695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:56.244472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662566+0000","last_change":"2023-11-24T07:58:38.669302+0000","last_active":"2023-11-24T07:58:39.662566+0000","last_peered":"2023-11-24T07:58:39.662566+0000","last_clean":"2023-11-24T07:58:39.662566+0000","last_became_active":"2023-11-24T07:58:38.669023+0000","last_became_peered":"2023-11-24T07:58:38.669023+0000","last_unstale":"2023-11-24T07:58:39.662566+0000","last_undegraded":"2023-11-24T07:58:39.662566+0000","last_fullsized":"2023-11-24T07:58:39.662566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210236+0000","last_change":"2023-11-24T07:58:37.659022+0000","last_active":"2023-11-24T07:58:40.210236+0000","last_peered":"2023-11-24T07:58:40.210236+0000","last_clean":"2023-11-24T07:58:40.210236+0000","last_became_active":"2023-11-24T07:58:37.658698+0000","last_became_peered":"2023-11-24T07:58:37.658698+0000","last_unstale":"2023-11-24T07:58:40.210236+0000","last_undegraded":"2023-11-24T07:58:40.210236+0000","last_fullsized":"2023-11-24T07:58:40.210236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662404+0000","last_change":"2023-11-24T07:58:38.669294+0000","last_active":"2023-11-24T07:58:39.662404+0000","last_peered":"2023-11-24T07:58:39.662404+0000","last_clean":"2023-11-24T07:58:39.662404+0000","last_became_active":"2023-11-24T07:58:38.668636+0000","last_became_peered":"2023-11-24T07:58:38.668636+0000","last_unstale":"2023-11-24T07:58:39.662404+0000","last_undegraded":"2023-11-24T07:58:39.662404+0000","last_fullsized":"2023-11-24T07:58:39.662404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655621+0000","last_change":"2023-11-24T07:58:37.655113+0000","last_active":"2023-11-24T07:58:39.655621+0000","last_peered":"2023-11-24T07:58:39.655621+0000","last_clean":"2023-11-24T07:58:39.655621+0000","last_became_active":"2023-11-24T07:58:37.654784+0000","last_became_peered":"2023-11-24T07:58:37.654784+0000","last_unstale":"2023-11-24T07:58:39.655621+0000","last_undegraded":"2023-11-24T07:58:39.655621+0000","last_fullsized":"2023-11-24T07:58:39.655621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209899+0000","last_change":"2023-11-24T07:58:38.653315+0000","last_active":"2023-11-24T07:58:40.209899+0000","last_peered":"2023-11-24T07:58:40.209899+0000","last_clean":"2023-11-24T07:58:40.209899+0000","last_became_active":"2023-11-24T07:58:38.653027+0000","last_became_peered":"2023-11-24T07:58:38.653027+0000","last_unstale":"2023-11-24T07:58:40.209899+0000","last_undegraded":"2023-11-24T07:58:40.209899+0000","last_fullsized":"2023-11-24T07:58:40.209899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:03.244592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660984+0000","last_change":"2023-11-24T07:58:37.660782+0000","last_active":"2023-11-24T07:58:39.660984+0000","last_peered":"2023-11-24T07:58:39.660984+0000","last_clean":"2023-11-24T07:58:39.660984+0000","last_became_active":"2023-11-24T07:58:37.660419+0000","last_became_peered":"2023-11-24T07:58:37.660419+0000","last_unstale":"2023-11-24T07:58:39.660984+0000","last_undegraded":"2023-11-24T07:58:39.660984+0000","last_fullsized":"2023-11-24T07:58:39.660984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670254+0000","last_change":"2023-11-24T07:58:38.667006+0000","last_active":"2023-11-24T07:58:39.670254+0000","last_peered":"2023-11-24T07:58:39.670254+0000","last_clean":"2023-11-24T07:58:39.670254+0000","last_became_active":"2023-11-24T07:58:38.665254+0000","last_became_peered":"2023-11-24T07:58:38.665254+0000","last_unstale":"2023-11-24T07:58:39.670254+0000","last_undegraded":"2023-11-24T07:58:39.670254+0000","last_fullsized":"2023-11-24T07:58:39.670254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655425+0000","last_change":"2023-11-24T07:58:37.660588+0000","last_active":"2023-11-24T07:58:39.655425+0000","last_peered":"2023-11-24T07:58:39.655425+0000","last_clean":"2023-11-24T07:58:39.655425+0000","last_became_active":"2023-11-24T07:58:37.660295+0000","last_became_peered":"2023-11-24T07:58:37.660295+0000","last_unstale":"2023-11-24T07:58:39.655425+0000","last_undegraded":"2023-11-24T07:58:39.655425+0000","last_fullsized":"2023-11-24T07:58:39.655425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655528+0000","last_change":"2023-11-24T07:58:38.671305+0000","last_active":"2023-11-24T07:58:39.655528+0000","last_peered":"2023-11-24T07:58:39.655528+0000","last_clean":"2023-11-24T07:58:39.655528+0000","last_became_active":"2023-11-24T07:58:38.670974+0000","last_became_peered":"2023-11-24T07:58:38.670974+0000","last_unstale":"2023-11-24T07:58:39.655528+0000","last_undegraded":"2023-11-24T07:58:39.655528+0000","last_fullsized":"2023-11-24T07:58:39.655528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661837+0000","last_change":"2023-11-24T07:58:37.658192+0000","last_active":"2023-11-24T07:58:39.661837+0000","last_peered":"2023-11-24T07:58:39.661837+0000","last_clean":"2023-11-24T07:58:39.661837+0000","last_became_active":"2023-11-24T07:58:37.657283+0000","last_became_peered":"2023-11-24T07:58:37.657283+0000","last_unstale":"2023-11-24T07:58:39.661837+0000","last_undegraded":"2023-11-24T07:58:39.661837+0000","last_fullsized":"2023-11-24T07:58:39.661837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209719+0000","last_change":"2023-11-24T07:58:38.655330+0000","last_active":"2023-11-24T07:58:40.209719+0000","last_peered":"2023-11-24T07:58:40.209719+0000","last_clean":"2023-11-24T07:58:40.209719+0000","last_became_active":"2023-11-24T07:58:38.654988+0000","last_became_peered":"2023-11-24T07:58:38.654988+0000","last_unstale":"2023-11-24T07:58:40.209719+0000","last_undegraded":"2023-11-24T07:58:40.209719+0000","last_fullsized":"2023-11-24T07:58:40.209719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661550+0000","last_change":"2023-11-24T07:58:37.663203+0000","last_active":"2023-11-24T07:58:39.661550+0000","last_peered":"2023-11-24T07:58:39.661550+0000","last_clean":"2023-11-24T07:58:39.661550+0000","last_became_active":"2023-11-24T07:58:37.662842+0000","last_became_peered":"2023-11-24T07:58:37.662842+0000","last_unstale":"2023-11-24T07:58:39.661550+0000","last_undegraded":"2023-11-24T07:58:39.661550+0000","last_fullsized":"2023-11-24T07:58:39.661550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662636+0000","last_change":"2023-11-24T07:58:38.662184+0000","last_active":"2023-11-24T07:58:39.662636+0000","last_peered":"2023-11-24T07:58:39.662636+0000","last_clean":"2023-11-24T07:58:39.662636+0000","last_became_active":"2023-11-24T07:58:38.661870+0000","last_became_peered":"2023-11-24T07:58:38.661870+0000","last_unstale":"2023-11-24T07:58:39.662636+0000","last_undegraded":"2023-11-24T07:58:39.662636+0000","last_fullsized":"2023-11-24T07:58:39.662636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662519+0000","last_change":"2023-11-24T07:58:37.663425+0000","last_active":"2023-11-24T07:58:39.662519+0000","last_peered":"2023-11-24T07:58:39.662519+0000","last_clean":"2023-11-24T07:58:39.662519+0000","last_became_active":"2023-11-24T07:58:37.663165+0000","last_became_peered":"2023-11-24T07:58:37.663165+0000","last_unstale":"2023-11-24T07:58:39.662519+0000","last_undegraded":"2023-11-24T07:58:39.662519+0000","last_fullsized":"2023-11-24T07:58:39.662519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670128+0000","last_change":"2023-11-24T07:58:38.669831+0000","last_active":"2023-11-24T07:58:39.670128+0000","last_peered":"2023-11-24T07:58:39.670128+0000","last_clean":"2023-11-24T07:58:39.670128+0000","last_became_active":"2023-11-24T07:58:38.669524+0000","last_became_peered":"2023-11-24T07:58:38.669524+0000","last_unstale":"2023-11-24T07:58:39.670128+0000","last_undegraded":"2023-11-24T07:58:39.670128+0000","last_fullsized":"2023-11-24T07:58:39.670128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209893+0000","last_change":"2023-11-24T07:58:37.656248+0000","last_active":"2023-11-24T07:58:40.209893+0000","last_peered":"2023-11-24T07:58:40.209893+0000","last_clean":"2023-11-24T07:58:40.209893+0000","last_became_active":"2023-11-24T07:58:37.655538+0000","last_became_peered":"2023-11-24T07:58:37.655538+0000","last_unstale":"2023-11-24T07:58:40.209893+0000","last_undegraded":"2023-11-24T07:58:40.209893+0000","last_fullsized":"2023-11-24T07:58:40.209893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210389+0000","last_change":"2023-11-24T07:58:38.660386+0000","last_active":"2023-11-24T07:58:40.210389+0000","last_peered":"2023-11-24T07:58:40.210389+0000","last_clean":"2023-11-24T07:58:40.210389+0000","last_became_active":"2023-11-24T07:58:38.659454+0000","last_became_peered":"2023-11-24T07:58:38.659454+0000","last_unstale":"2023-11-24T07:58:40.210389+0000","last_undegraded":"2023-11-24T07:58:40.210389+0000","last_fullsized":"2023-11-24T07:58:40.210389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661142+0000","last_change":"2023-11-24T07:58:37.663115+0000","last_active":"2023-11-24T07:58:39.661142+0000","last_peered":"2023-11-24T07:58:39.661142+0000","last_clean":"2023-11-24T07:58:39.661142+0000","last_became_active":"2023-11-24T07:58:37.662759+0000","last_became_peered":"2023-11-24T07:58:37.662759+0000","last_unstale":"2023-11-24T07:58:39.661142+0000","last_undegraded":"2023-11-24T07:58:39.661142+0000","last_fullsized":"2023-11-24T07:58:39.661142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210514+0000","last_change":"2023-11-24T07:58:38.677184+0000","last_active":"2023-11-24T07:58:40.210514+0000","last_peered":"2023-11-24T07:58:40.210514+0000","last_clean":"2023-11-24T07:58:40.210514+0000","last_became_active":"2023-11-24T07:58:38.676729+0000","last_became_peered":"2023-11-24T07:58:38.676729+0000","last_unstale":"2023-11-24T07:58:40.210514+0000","last_undegraded":"2023-11-24T07:58:40.210514+0000","last_fullsized":"2023-11-24T07:58:40.210514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670164+0000","last_change":"2023-11-24T07:58:37.659853+0000","last_active":"2023-11-24T07:58:39.670164+0000","last_peered":"2023-11-24T07:58:39.670164+0000","last_clean":"2023-11-24T07:58:39.670164+0000","last_became_active":"2023-11-24T07:58:37.659563+0000","last_became_peered":"2023-11-24T07:58:37.659563+0000","last_unstale":"2023-11-24T07:58:39.670164+0000","last_undegraded":"2023-11-24T07:58:39.670164+0000","last_fullsized":"2023-11-24T07:58:39.670164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670259+0000","last_change":"2023-11-24T07:58:38.668668+0000","last_active":"2023-11-24T07:58:39.670259+0000","last_peered":"2023-11-24T07:58:39.670259+0000","last_clean":"2023-11-24T07:58:39.670259+0000","last_became_active":"2023-11-24T07:58:38.668181+0000","last_became_peered":"2023-11-24T07:58:38.668181+0000","last_unstale":"2023-11-24T07:58:39.670259+0000","last_undegraded":"2023-11-24T07:58:39.670259+0000","last_fullsized":"2023-11-24T07:58:39.670259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662643+0000","last_change":"2023-11-24T07:58:37.656567+0000","last_active":"2023-11-24T07:58:39.662643+0000","last_peered":"2023-11-24T07:58:39.662643+0000","last_clean":"2023-11-24T07:58:39.662643+0000","last_became_active":"2023-11-24T07:58:37.656024+0000","last_became_peered":"2023-11-24T07:58:37.656024+0000","last_unstale":"2023-11-24T07:58:39.662643+0000","last_undegraded":"2023-11-24T07:58:39.662643+0000","last_fullsized":"2023-11-24T07:58:39.662643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210630+0000","last_change":"2023-11-24T07:58:38.664336+0000","last_active":"2023-11-24T07:58:40.210630+0000","last_peered":"2023-11-24T07:58:40.210630+0000","last_clean":"2023-11-24T07:58:40.210630+0000","last_became_active":"2023-11-24T07:58:38.664080+0000","last_became_peered":"2023-11-24T07:58:38.664080+0000","last_unstale":"2023-11-24T07:58:40.210630+0000","last_undegraded":"2023-11-24T07:58:40.210630+0000","last_fullsized":"2023-11-24T07:58:40.210630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660834+0000","last_change":"2023-11-24T07:58:37.660791+0000","last_active":"2023-11-24T07:58:39.660834+0000","last_peered":"2023-11-24T07:58:39.660834+0000","last_clean":"2023-11-24T07:58:39.660834+0000","last_became_active":"2023-11-24T07:58:37.660488+0000","last_became_peered":"2023-11-24T07:58:37.660488+0000","last_unstale":"2023-11-24T07:58:39.660834+0000","last_undegraded":"2023-11-24T07:58:39.660834+0000","last_fullsized":"2023-11-24T07:58:39.660834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660799+0000","last_change":"2023-11-24T07:58:38.670298+0000","last_active":"2023-11-24T07:58:39.660799+0000","last_peered":"2023-11-24T07:58:39.660799+0000","last_clean":"2023-11-24T07:58:39.660799+0000","last_became_active":"2023-11-24T07:58:38.669956+0000","last_became_peered":"2023-11-24T07:58:38.669956+0000","last_unstale":"2023-11-24T07:58:39.660799+0000","last_undegraded":"2023-11-24T07:58:39.660799+0000","last_fullsized":"2023-11-24T07:58:39.660799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209457+0000","last_change":"2023-11-24T07:58:37.659643+0000","last_active":"2023-11-24T07:58:40.209457+0000","last_peered":"2023-11-24T07:58:40.209457+0000","last_clean":"2023-11-24T07:58:40.209457+0000","last_became_active":"2023-11-24T07:58:37.659317+0000","last_became_peered":"2023-11-24T07:58:37.659317+0000","last_unstale":"2023-11-24T07:58:40.209457+0000","last_undegraded":"2023-11-24T07:58:40.209457+0000","last_fullsized":"2023-11-24T07:58:40.209457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:32.387984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209982+0000","last_change":"2023-11-24T07:58:38.667271+0000","last_active":"2023-11-24T07:58:40.209982+0000","last_peered":"2023-11-24T07:58:40.209982+0000","last_clean":"2023-11-24T07:58:40.209982+0000","last_became_active":"2023-11-24T07:58:38.666904+0000","last_became_peered":"2023-11-24T07:58:38.666904+0000","last_unstale":"2023-11-24T07:58:40.209982+0000","last_undegraded":"2023-11-24T07:58:40.209982+0000","last_fullsized":"2023-11-24T07:58:40.209982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210573+0000","last_change":"2023-11-24T07:58:37.657085+0000","last_active":"2023-11-24T07:58:40.210573+0000","last_peered":"2023-11-24T07:58:40.210573+0000","last_clean":"2023-11-24T07:58:40.210573+0000","last_became_active":"2023-11-24T07:58:37.656676+0000","last_became_peered":"2023-11-24T07:58:37.656676+0000","last_unstale":"2023-11-24T07:58:40.210573+0000","last_undegraded":"2023-11-24T07:58:40.210573+0000","last_fullsized":"2023-11-24T07:58:40.210573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:45.701317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210306+0000","last_change":"2023-11-24T07:58:38.664417+0000","last_active":"2023-11-24T07:58:40.210306+0000","last_peered":"2023-11-24T07:58:40.210306+0000","last_clean":"2023-11-24T07:58:40.210306+0000","last_became_active":"2023-11-24T07:58:38.664161+0000","last_became_peered":"2023-11-24T07:58:38.664161+0000","last_unstale":"2023-11-24T07:58:40.210306+0000","last_undegraded":"2023-11-24T07:58:40.210306+0000","last_fullsized":"2023-11-24T07:58:40.210306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209807+0000","last_change":"2023-11-24T07:58:37.660866+0000","last_active":"2023-11-24T07:58:40.209807+0000","last_peered":"2023-11-24T07:58:40.209807+0000","last_clean":"2023-11-24T07:58:40.209807+0000","last_became_active":"2023-11-24T07:58:37.660577+0000","last_became_peered":"2023-11-24T07:58:37.660577+0000","last_unstale":"2023-11-24T07:58:40.209807+0000","last_undegraded":"2023-11-24T07:58:40.209807+0000","last_fullsized":"2023-11-24T07:58:40.209807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662441+0000","last_change":"2023-11-24T07:58:38.674685+0000","last_active":"2023-11-24T07:58:39.662441+0000","last_peered":"2023-11-24T07:58:39.662441+0000","last_clean":"2023-11-24T07:58:39.662441+0000","last_became_active":"2023-11-24T07:58:38.674366+0000","last_became_peered":"2023-11-24T07:58:38.674366+0000","last_unstale":"2023-11-24T07:58:39.662441+0000","last_undegraded":"2023-11-24T07:58:39.662441+0000","last_fullsized":"2023-11-24T07:58:39.662441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662580+0000","last_change":"2023-11-24T07:58:37.655735+0000","last_active":"2023-11-24T07:58:39.662580+0000","last_peered":"2023-11-24T07:58:39.662580+0000","last_clean":"2023-11-24T07:58:39.662580+0000","last_became_active":"2023-11-24T07:58:37.655444+0000","last_became_peered":"2023-11-24T07:58:37.655444+0000","last_unstale":"2023-11-24T07:58:39.662580+0000","last_undegraded":"2023-11-24T07:58:39.662580+0000","last_fullsized":"2023-11-24T07:58:39.662580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:10.569901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210250+0000","last_change":"2023-11-24T07:58:38.667440+0000","last_active":"2023-11-24T07:58:40.210250+0000","last_peered":"2023-11-24T07:58:40.210250+0000","last_clean":"2023-11-24T07:58:40.210250+0000","last_became_active":"2023-11-24T07:58:38.666914+0000","last_became_peered":"2023-11-24T07:58:38.666914+0000","last_unstale":"2023-11-24T07:58:40.210250+0000","last_undegraded":"2023-11-24T07:58:40.210250+0000","last_fullsized":"2023-11-24T07:58:40.210250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:19.761593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209468+0000","last_change":"2023-11-24T07:58:38.655018+0000","last_active":"2023-11-24T07:58:40.209468+0000","last_peered":"2023-11-24T07:58:40.209468+0000","last_clean":"2023-11-24T07:58:40.209468+0000","last_became_active":"2023-11-24T07:58:38.654604+0000","last_became_peered":"2023-11-24T07:58:38.654604+0000","last_unstale":"2023-11-24T07:58:40.209468+0000","last_undegraded":"2023-11-24T07:58:40.209468+0000","last_fullsized":"2023-11-24T07:58:40.209468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209541+0000","last_change":"2023-11-24T07:58:37.660113+0000","last_active":"2023-11-24T07:58:40.209541+0000","last_peered":"2023-11-24T07:58:40.209541+0000","last_clean":"2023-11-24T07:58:40.209541+0000","last_became_active":"2023-11-24T07:58:37.659747+0000","last_became_peered":"2023-11-24T07:58:37.659747+0000","last_unstale":"2023-11-24T07:58:40.209541+0000","last_undegraded":"2023-11-24T07:58:40.209541+0000","last_fullsized":"2023-11-24T07:58:40.209541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:02.242362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655662+0000","last_change":"2023-11-24T07:58:38.670798+0000","last_active":"2023-11-24T07:58:39.655662+0000","last_peered":"2023-11-24T07:58:39.655662+0000","last_clean":"2023-11-24T07:58:39.655662+0000","last_became_active":"2023-11-24T07:58:38.670464+0000","last_became_peered":"2023-11-24T07:58:38.670464+0000","last_unstale":"2023-11-24T07:58:39.655662+0000","last_undegraded":"2023-11-24T07:58:39.655662+0000","last_fullsized":"2023-11-24T07:58:39.655662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662521+0000","last_change":"2023-11-24T07:58:37.661568+0000","last_active":"2023-11-24T07:58:39.662521+0000","last_peered":"2023-11-24T07:58:39.662521+0000","last_clean":"2023-11-24T07:58:39.662521+0000","last_became_active":"2023-11-24T07:58:37.661211+0000","last_became_peered":"2023-11-24T07:58:37.661211+0000","last_unstale":"2023-11-24T07:58:39.662521+0000","last_undegraded":"2023-11-24T07:58:39.662521+0000","last_fullsized":"2023-11-24T07:58:39.662521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:58:38.650810+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:58:38.650810+0000","last_peered":"2023-11-24T07:58:38.650810+0000","last_clean":"2023-11-24T07:58:38.650810+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:58:38.650810+0000","last_undegraded":"2023-11-24T07:58:38.650810+0000","last_fullsized":"2023-11-24T07:58:38.650810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669820+0000","last_change":"2023-11-24T07:58:38.667012+0000","last_active":"2023-11-24T07:58:39.669820+0000","last_peered":"2023-11-24T07:58:39.669820+0000","last_clean":"2023-11-24T07:58:39.669820+0000","last_became_active":"2023-11-24T07:58:38.666604+0000","last_became_peered":"2023-11-24T07:58:38.666604+0000","last_unstale":"2023-11-24T07:58:39.669820+0000","last_undegraded":"2023-11-24T07:58:39.669820+0000","last_fullsized":"2023-11-24T07:58:39.669820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.085407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209908+0000","last_change":"2023-11-24T07:58:37.655598+0000","last_active":"2023-11-24T07:58:40.209908+0000","last_peered":"2023-11-24T07:58:40.209908+0000","last_clean":"2023-11-24T07:58:40.209908+0000","last_became_active":"2023-11-24T07:58:37.655321+0000","last_became_peered":"2023-11-24T07:58:37.655321+0000","last_unstale":"2023-11-24T07:58:40.209908+0000","last_undegraded":"2023-11-24T07:58:40.209908+0000","last_fullsized":"2023-11-24T07:58:40.209908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210609+0000","last_change":"2023-11-24T07:58:37.659615+0000","last_active":"2023-11-24T07:58:40.210609+0000","last_peered":"2023-11-24T07:58:40.210609+0000","last_clean":"2023-11-24T07:58:40.210609+0000","last_became_active":"2023-11-24T07:58:37.659362+0000","last_became_peered":"2023-11-24T07:58:37.659362+0000","last_unstale":"2023-11-24T07:58:40.210609+0000","last_undegraded":"2023-11-24T07:58:40.210609+0000","last_fullsized":"2023-11-24T07:58:40.210609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209722+0000","last_change":"2023-11-24T07:58:38.655615+0000","last_active":"2023-11-24T07:58:40.209722+0000","last_peered":"2023-11-24T07:58:40.209722+0000","last_clean":"2023-11-24T07:58:40.209722+0000","last_became_active":"2023-11-24T07:58:38.654516+0000","last_became_peered":"2023-11-24T07:58:38.654516+0000","last_unstale":"2023-11-24T07:58:40.209722+0000","last_undegraded":"2023-11-24T07:58:40.209722+0000","last_fullsized":"2023-11-24T07:58:40.209722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209797+0000","last_change":"2023-11-24T07:58:37.663218+0000","last_active":"2023-11-24T07:58:40.209797+0000","last_peered":"2023-11-24T07:58:40.209797+0000","last_clean":"2023-11-24T07:58:40.209797+0000","last_became_active":"2023-11-24T07:58:37.662867+0000","last_became_peered":"2023-11-24T07:58:37.662867+0000","last_unstale":"2023-11-24T07:58:40.209797+0000","last_undegraded":"2023-11-24T07:58:40.209797+0000","last_fullsized":"2023-11-24T07:58:40.209797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656341+0000","last_change":"2023-11-24T07:58:38.670569+0000","last_active":"2023-11-24T07:58:39.656341+0000","last_peered":"2023-11-24T07:58:39.656341+0000","last_clean":"2023-11-24T07:58:39.656341+0000","last_became_active":"2023-11-24T07:58:38.670239+0000","last_became_peered":"2023-11-24T07:58:38.670239+0000","last_unstale":"2023-11-24T07:58:39.656341+0000","last_undegraded":"2023-11-24T07:58:39.656341+0000","last_fullsized":"2023-11-24T07:58:39.656341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661516+0000","last_change":"2023-11-24T07:58:37.667212+0000","last_active":"2023-11-24T07:58:39.661516+0000","last_peered":"2023-11-24T07:58:39.661516+0000","last_clean":"2023-11-24T07:58:39.661516+0000","last_became_active":"2023-11-24T07:58:37.666944+0000","last_became_peered":"2023-11-24T07:58:37.666944+0000","last_unstale":"2023-11-24T07:58:39.661516+0000","last_undegraded":"2023-11-24T07:58:39.661516+0000","last_fullsized":"2023-11-24T07:58:39.661516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670071+0000","last_change":"2023-11-24T07:58:38.660314+0000","last_active":"2023-11-24T07:58:39.670071+0000","last_peered":"2023-11-24T07:58:39.670071+0000","last_clean":"2023-11-24T07:58:39.670071+0000","last_became_active":"2023-11-24T07:58:38.660005+0000","last_became_peered":"2023-11-24T07:58:38.660005+0000","last_unstale":"2023-11-24T07:58:39.670071+0000","last_undegraded":"2023-11-24T07:58:39.670071+0000","last_fullsized":"2023-11-24T07:58:39.670071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209750+0000","last_change":"2023-11-24T07:58:37.667045+0000","last_active":"2023-11-24T07:58:40.209750+0000","last_peered":"2023-11-24T07:58:40.209750+0000","last_clean":"2023-11-24T07:58:40.209750+0000","last_became_active":"2023-11-24T07:58:37.666766+0000","last_became_peered":"2023-11-24T07:58:37.666766+0000","last_unstale":"2023-11-24T07:58:40.209750+0000","last_undegraded":"2023-11-24T07:58:40.209750+0000","last_fullsized":"2023-11-24T07:58:40.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209814+0000","last_change":"2023-11-24T07:58:38.652046+0000","last_active":"2023-11-24T07:58:40.209814+0000","last_peered":"2023-11-24T07:58:40.209814+0000","last_clean":"2023-11-24T07:58:40.209814+0000","last_became_active":"2023-11-24T07:58:38.651762+0000","last_became_peered":"2023-11-24T07:58:38.651762+0000","last_unstale":"2023-11-24T07:58:40.209814+0000","last_undegraded":"2023-11-24T07:58:40.209814+0000","last_fullsized":"2023-11-24T07:58:40.209814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:28.237034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209556+0000","last_change":"2023-11-24T07:58:37.655838+0000","last_active":"2023-11-24T07:58:40.209556+0000","last_peered":"2023-11-24T07:58:40.209556+0000","last_clean":"2023-11-24T07:58:40.209556+0000","last_became_active":"2023-11-24T07:58:37.655553+0000","last_became_peered":"2023-11-24T07:58:37.655553+0000","last_unstale":"2023-11-24T07:58:40.209556+0000","last_undegraded":"2023-11-24T07:58:40.209556+0000","last_fullsized":"2023-11-24T07:58:40.209556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655413+0000","last_change":"2023-11-24T07:58:38.655539+0000","last_active":"2023-11-24T07:58:39.655413+0000","last_peered":"2023-11-24T07:58:39.655413+0000","last_clean":"2023-11-24T07:58:39.655413+0000","last_became_active":"2023-11-24T07:58:38.655169+0000","last_became_peered":"2023-11-24T07:58:38.655169+0000","last_unstale":"2023-11-24T07:58:39.655413+0000","last_undegraded":"2023-11-24T07:58:39.655413+0000","last_fullsized":"2023-11-24T07:58:39.655413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209712+0000","last_change":"2023-11-24T07:58:37.655841+0000","last_active":"2023-11-24T07:58:40.209712+0000","last_peered":"2023-11-24T07:58:40.209712+0000","last_clean":"2023-11-24T07:58:40.209712+0000","last_became_active":"2023-11-24T07:58:37.655487+0000","last_became_peered":"2023-11-24T07:58:37.655487+0000","last_unstale":"2023-11-24T07:58:40.209712+0000","last_undegraded":"2023-11-24T07:58:40.209712+0000","last_fullsized":"2023-11-24T07:58:40.209712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655699+0000","last_change":"2023-11-24T07:58:38.671051+0000","last_active":"2023-11-24T07:58:39.655699+0000","last_peered":"2023-11-24T07:58:39.655699+0000","last_clean":"2023-11-24T07:58:39.655699+0000","last_became_active":"2023-11-24T07:58:38.670571+0000","last_became_peered":"2023-11-24T07:58:38.670571+0000","last_unstale":"2023-11-24T07:58:39.655699+0000","last_undegraded":"2023-11-24T07:58:39.655699+0000","last_fullsized":"2023-11-24T07:58:39.655699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210427+0000","last_change":"2023-11-24T07:58:37.655295+0000","last_active":"2023-11-24T07:58:40.210427+0000","last_peered":"2023-11-24T07:58:40.210427+0000","last_clean":"2023-11-24T07:58:40.210427+0000","last_became_active":"2023-11-24T07:58:37.654767+0000","last_became_peered":"2023-11-24T07:58:37.654767+0000","last_unstale":"2023-11-24T07:58:40.210427+0000","last_undegraded":"2023-11-24T07:58:40.210427+0000","last_fullsized":"2023-11-24T07:58:40.210427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660877+0000","last_change":"2023-11-24T07:58:38.662792+0000","last_active":"2023-11-24T07:58:39.660877+0000","last_peered":"2023-11-24T07:58:39.660877+0000","last_clean":"2023-11-24T07:58:39.660877+0000","last_became_active":"2023-11-24T07:58:38.662492+0000","last_became_peered":"2023-11-24T07:58:38.662492+0000","last_unstale":"2023-11-24T07:58:39.660877+0000","last_undegraded":"2023-11-24T07:58:39.660877+0000","last_fullsized":"2023-11-24T07:58:39.660877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210083+0000","last_change":"2023-11-24T07:58:37.660634+0000","last_active":"2023-11-24T07:58:40.210083+0000","last_peered":"2023-11-24T07:58:40.210083+0000","last_clean":"2023-11-24T07:58:40.210083+0000","last_became_active":"2023-11-24T07:58:37.660340+0000","last_became_peered":"2023-11-24T07:58:37.660340+0000","last_unstale":"2023-11-24T07:58:40.210083+0000","last_undegraded":"2023-11-24T07:58:40.210083+0000","last_fullsized":"2023-11-24T07:58:40.210083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:04.555457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210612+0000","last_change":"2023-11-24T07:58:38.672774+0000","last_active":"2023-11-24T07:58:40.210612+0000","last_peered":"2023-11-24T07:58:40.210612+0000","last_clean":"2023-11-24T07:58:40.210612+0000","last_became_active":"2023-11-24T07:58:38.672498+0000","last_became_peered":"2023-11-24T07:58:38.672498+0000","last_unstale":"2023-11-24T07:58:40.210612+0000","last_undegraded":"2023-11-24T07:58:40.210612+0000","last_fullsized":"2023-11-24T07:58:40.210612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209839+0000","last_change":"2023-11-24T07:58:37.653547+0000","last_active":"2023-11-24T07:58:40.209839+0000","last_peered":"2023-11-24T07:58:40.209839+0000","last_clean":"2023-11-24T07:58:40.209839+0000","last_became_active":"2023-11-24T07:58:37.653148+0000","last_became_peered":"2023-11-24T07:58:37.653148+0000","last_unstale":"2023-11-24T07:58:40.209839+0000","last_undegraded":"2023-11-24T07:58:40.209839+0000","last_fullsized":"2023-11-24T07:58:40.209839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662278+0000","last_change":"2023-11-24T07:58:38.676919+0000","last_active":"2023-11-24T07:58:39.662278+0000","last_peered":"2023-11-24T07:58:39.662278+0000","last_clean":"2023-11-24T07:58:39.662278+0000","last_became_active":"2023-11-24T07:58:38.676622+0000","last_became_peered":"2023-11-24T07:58:38.676622+0000","last_unstale":"2023-11-24T07:58:39.662278+0000","last_undegraded":"2023-11-24T07:58:39.662278+0000","last_fullsized":"2023-11-24T07:58:39.662278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210143+0000","last_change":"2023-11-24T07:58:37.659572+0000","last_active":"2023-11-24T07:58:40.210143+0000","last_peered":"2023-11-24T07:58:40.210143+0000","last_clean":"2023-11-24T07:58:40.210143+0000","last_became_active":"2023-11-24T07:58:37.659273+0000","last_became_peered":"2023-11-24T07:58:37.659273+0000","last_unstale":"2023-11-24T07:58:40.210143+0000","last_undegraded":"2023-11-24T07:58:40.210143+0000","last_fullsized":"2023-11-24T07:58:40.210143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210569+0000","last_change":"2023-11-24T07:58:38.673691+0000","last_active":"2023-11-24T07:58:40.210569+0000","last_peered":"2023-11-24T07:58:40.210569+0000","last_clean":"2023-11-24T07:58:40.210569+0000","last_became_active":"2023-11-24T07:58:38.673092+0000","last_became_peered":"2023-11-24T07:58:38.673092+0000","last_unstale":"2023-11-24T07:58:40.210569+0000","last_undegraded":"2023-11-24T07:58:40.210569+0000","last_fullsized":"2023-11-24T07:58:40.210569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661147+0000","last_change":"2023-11-24T07:58:38.670271+0000","last_active":"2023-11-24T07:58:39.661147+0000","last_peered":"2023-11-24T07:58:39.661147+0000","last_clean":"2023-11-24T07:58:39.661147+0000","last_became_active":"2023-11-24T07:58:38.669965+0000","last_became_peered":"2023-11-24T07:58:38.669965+0000","last_unstale":"2023-11-24T07:58:39.661147+0000","last_undegraded":"2023-11-24T07:58:39.661147+0000","last_fullsized":"2023-11-24T07:58:39.661147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209683+0000","last_change":"2023-11-24T07:58:38.663275+0000","last_active":"2023-11-24T07:58:40.209683+0000","last_peered":"2023-11-24T07:58:40.209683+0000","last_clean":"2023-11-24T07:58:40.209683+0000","last_became_active":"2023-11-24T07:58:38.662928+0000","last_became_peered":"2023-11-24T07:58:38.662928+0000","last_unstale":"2023-11-24T07:58:40.209683+0000","last_undegraded":"2023-11-24T07:58:40.209683+0000","last_fullsized":"2023-11-24T07:58:40.209683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662492+0000","last_change":"2023-11-24T07:58:38.674085+0000","last_active":"2023-11-24T07:58:39.662492+0000","last_peered":"2023-11-24T07:58:39.662492+0000","last_clean":"2023-11-24T07:58:39.662492+0000","last_became_active":"2023-11-24T07:58:38.673780+0000","last_became_peered":"2023-11-24T07:58:38.673780+0000","last_unstale":"2023-11-24T07:58:39.662492+0000","last_undegraded":"2023-11-24T07:58:39.662492+0000","last_fullsized":"2023-11-24T07:58:39.662492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670095+0000","last_change":"2023-11-24T07:58:38.676742+0000","last_active":"2023-11-24T07:58:39.670095+0000","last_peered":"2023-11-24T07:58:39.670095+0000","last_clean":"2023-11-24T07:58:39.670095+0000","last_became_active":"2023-11-24T07:58:38.676422+0000","last_became_peered":"2023-11-24T07:58:38.676422+0000","last_unstale":"2023-11-24T07:58:39.670095+0000","last_undegraded":"2023-11-24T07:58:39.670095+0000","last_fullsized":"2023-11-24T07:58:39.670095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655920+0000","last_change":"2023-11-24T07:58:38.674946+0000","last_active":"2023-11-24T07:58:39.655920+0000","last_peered":"2023-11-24T07:58:39.655920+0000","last_clean":"2023-11-24T07:58:39.655920+0000","last_became_active":"2023-11-24T07:58:38.674458+0000","last_became_peered":"2023-11-24T07:58:38.674458+0000","last_unstale":"2023-11-24T07:58:39.655920+0000","last_undegraded":"2023-11-24T07:58:39.655920+0000","last_fullsized":"2023-11-24T07:58:39.655920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209765+0000","last_change":"2023-11-24T07:58:38.656486+0000","last_active":"2023-11-24T07:58:40.209765+0000","last_peered":"2023-11-24T07:58:40.209765+0000","last_clean":"2023-11-24T07:58:40.209765+0000","last_became_active":"2023-11-24T07:58:38.654844+0000","last_became_peered":"2023-11-24T07:58:38.654844+0000","last_unstale":"2023-11-24T07:58:40.209765+0000","last_undegraded":"2023-11-24T07:58:40.209765+0000","last_fullsized":"2023-11-24T07:58:40.209765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661231+0000","last_change":"2023-11-24T07:58:38.666517+0000","last_active":"2023-11-24T07:58:39.661231+0000","last_peered":"2023-11-24T07:58:39.661231+0000","last_clean":"2023-11-24T07:58:39.661231+0000","last_became_active":"2023-11-24T07:58:38.665821+0000","last_became_peered":"2023-11-24T07:58:38.665821+0000","last_unstale":"2023-11-24T07:58:39.661231+0000","last_undegraded":"2023-11-24T07:58:39.661231+0000","last_fullsized":"2023-11-24T07:58:39.661231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656201+0000","last_change":"2023-11-24T07:58:38.672413+0000","last_active":"2023-11-24T07:58:39.656201+0000","last_peered":"2023-11-24T07:58:39.656201+0000","last_clean":"2023-11-24T07:58:39.656201+0000","last_became_active":"2023-11-24T07:58:38.672131+0000","last_became_peered":"2023-11-24T07:58:38.672131+0000","last_unstale":"2023-11-24T07:58:39.656201+0000","last_undegraded":"2023-11-24T07:58:39.656201+0000","last_fullsized":"2023-11-24T07:58:39.656201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662001+0000","last_change":"2023-11-24T07:58:38.672368+0000","last_active":"2023-11-24T07:58:39.662001+0000","last_peered":"2023-11-24T07:58:39.662001+0000","last_clean":"2023-11-24T07:58:39.662001+0000","last_became_active":"2023-11-24T07:58:38.672038+0000","last_became_peered":"2023-11-24T07:58:38.672038+0000","last_unstale":"2023-11-24T07:58:39.662001+0000","last_undegraded":"2023-11-24T07:58:39.662001+0000","last_fullsized":"2023-11-24T07:58:39.662001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655743+0000","last_change":"2023-11-24T07:58:38.671825+0000","last_active":"2023-11-24T07:58:39.655743+0000","last_peered":"2023-11-24T07:58:39.655743+0000","last_clean":"2023-11-24T07:58:39.655743+0000","last_became_active":"2023-11-24T07:58:38.671530+0000","last_became_peered":"2023-11-24T07:58:38.671530+0000","last_unstale":"2023-11-24T07:58:39.655743+0000","last_undegraded":"2023-11-24T07:58:39.655743+0000","last_fullsized":"2023-11-24T07:58:39.655743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662135+0000","last_change":"2023-11-24T07:58:38.663149+0000","last_active":"2023-11-24T07:58:39.662135+0000","last_peered":"2023-11-24T07:58:39.662135+0000","last_clean":"2023-11-24T07:58:39.662135+0000","last_became_active":"2023-11-24T07:58:38.662826+0000","last_became_peered":"2023-11-24T07:58:38.662826+0000","last_unstale":"2023-11-24T07:58:39.662135+0000","last_undegraded":"2023-11-24T07:58:39.662135+0000","last_fullsized":"2023-11-24T07:58:39.662135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670177+0000","last_change":"2023-11-24T07:58:38.673409+0000","last_active":"2023-11-24T07:58:39.670177+0000","last_peered":"2023-11-24T07:58:39.670177+0000","last_clean":"2023-11-24T07:58:39.670177+0000","last_became_active":"2023-11-24T07:58:38.673109+0000","last_became_peered":"2023-11-24T07:58:38.673109+0000","last_unstale":"2023-11-24T07:58:39.670177+0000","last_undegraded":"2023-11-24T07:58:39.670177+0000","last_fullsized":"2023-11-24T07:58:39.670177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661303+0000","last_change":"2023-11-24T07:58:38.666773+0000","last_active":"2023-11-24T07:58:39.661303+0000","last_peered":"2023-11-24T07:58:39.661303+0000","last_clean":"2023-11-24T07:58:39.661303+0000","last_became_active":"2023-11-24T07:58:38.666430+0000","last_became_peered":"2023-11-24T07:58:38.666430+0000","last_unstale":"2023-11-24T07:58:39.661303+0000","last_undegraded":"2023-11-24T07:58:39.661303+0000","last_fullsized":"2023-11-24T07:58:39.661303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:42.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.962+0000 7f329f9aa700 1 Processor -- start 2023-11-24T07:58:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.962+0000 7f329f9aa700 1 -- start start 2023-11-24T07:58:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329f9aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32980ce4d0 0x7f32980cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329f9aa700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32980d1a30 0x7f32980d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329f9aa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32980d0240 con 0x7f329815df10 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329f9aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32980d23c0 con 0x7f32980ce4d0 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329f9aa700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329806ee10 con 0x7f32980d1a30 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329cf45700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32980ce4d0 0x7f32980cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329cf45700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32980ce4d0 0x7f32980cfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56434/0 (socket says 172.21.15.32:56434) 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329cf45700 1 -- 172.21.15.32:0/3258375566 learned_addr learned my addr 172.21.15.32:0/3258375566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.963+0000 7f329df47700 1 --2- 172.21.15.32:0/3258375566 >> v2:172.21.15.123:3300/0 conn(0x7f32980d1a30 0x7f32980d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329cf45700 1 -- 172.21.15.32:0/3258375566 >> v2:172.21.15.123:3300/0 conn(0x7f32980d1a30 msgr2=0x7f32980d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329cf45700 1 --2- 172.21.15.32:0/3258375566 >> v2:172.21.15.123:3300/0 conn(0x7f32980d1a30 0x7f32980d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329cf45700 1 -- 172.21.15.32:0/3258375566 >> v1:172.21.15.123:6789/0 conn(0x7f329815df10 legacy=0x7f329806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329cf45700 1 -- 172.21.15.32:0/3258375566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329806c090 con 0x7f32980ce4d0 2023-11-24T07:58:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329cf45700 1 --2- 172.21.15.32:0/3258375566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980ce4d0 0x7f32980cfd00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f32980c56c0 tx=0x7f3294009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b4dad50059f9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f328ffff700 1 -- 172.21.15.32:0/3258375566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400b870 con 0x7f32980ce4d0 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f328ffff700 1 -- 172.21.15.32:0/3258375566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400b9d0 con 0x7f32980ce4d0 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f328ffff700 1 -- 172.21.15.32:0/3258375566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400bd60 con 0x7f32980ce4d0 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329f9aa700 1 -- 172.21.15.32:0/3258375566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980ce4d0 msgr2=0x7f32980cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329f9aa700 1 --2- 172.21.15.32:0/3258375566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980ce4d0 0x7f32980cfd00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f32980c56c0 tx=0x7f3294009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329f9aa700 1 -- 172.21.15.32:0/3258375566 shutdown_connections 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329f9aa700 1 --2- 172.21.15.32:0/3258375566 >> v2:172.21.15.123:3300/0 conn(0x7f32980d1a30 0x7f32980d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329f9aa700 1 --2- 172.21.15.32:0/3258375566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980ce4d0 0x7f32980cfd00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.964+0000 7f329f9aa700 1 -- 172.21.15.32:0/3258375566 >> 172.21.15.32:0/3258375566 conn(0x7f329809aa00 msgr2=0x7f329809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.965+0000 7f329f9aa700 1 -- 172.21.15.32:0/3258375566 shutdown_connections 2023-11-24T07:58:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.965+0000 7f329f9aa700 1 -- 172.21.15.32:0/3258375566 wait complete. 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.966+0000 7f329f9aa700 1 Processor -- start 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.966+0000 7f329f9aa700 1 -- start start 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329f9aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32980ce4d0 0x7f329814d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329f9aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 0x7f329814a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329f9aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f329815df10 0x7f329815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329f9aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32980b4de0 con 0x7f32980d1a30 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329f9aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32981f4ad0 con 0x7f32980ce4d0 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329f9aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329814dbd0 con 0x7f329815df10 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329cf45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 0x7f329814a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329cf45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 0x7f329814a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56440/0 (socket says 172.21.15.32:56440) 2023-11-24T07:58:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329cf45700 1 -- 172.21.15.32:0/2390230450 learned_addr learned my addr 172.21.15.32:0/2390230450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329d746700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32980ce4d0 0x7f329814d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.967+0000 7f329df47700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f329815df10 0x7f329815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329cf45700 1 -- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f329815df10 msgr2=0x7f329815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329cf45700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f329815df10 0x7f329815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329cf45700 1 -- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32980ce4d0 msgr2=0x7f329814d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329cf45700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32980ce4d0 0x7f329814d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329cf45700 1 -- 172.21.15.32:0/2390230450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294009550 con 0x7f32980d1a30 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329df47700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f329815df10 0x7f329815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329cf45700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 0x7f329814a2b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f3294004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294004540 con 0x7f32980d1a30 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32940036a0 con 0x7f32980d1a30 2023-11-24T07:58:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294003a10 con 0x7f32980d1a30 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32981f3f50 con 0x7f32980d1a30 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.968+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32981f4510 con 0x7f32980d1a30 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.969+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3298067f60 con 0x7f32980d1a30 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.970+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3294021070 con 0x7f32980d1a30 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.970+0000 7f328dffb700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f327c06aed0 0x7f327c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.970+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f329408ae90 con 0x7f32980d1a30 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.971+0000 7f329d746700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f327c06aed0 0x7f327c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.971+0000 7f329d746700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f327c06aed0 0x7f327c06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3288007fd0 tx=0x7f3288009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:42.974+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3294055fb0 con 0x7f32980d1a30 2023-11-24T07:58:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.174+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f329806c910 con 0x7f327c06aed0 2023-11-24T07:58:43.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.182+0000 7f328dffb700 1 -- 172.21.15.32:0/2390230450 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264658 (secure 0 0 0) 0x7f3288001cc0 con 0x7f327c06aed0 2023-11-24T07:58:43.187 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:43.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f327c06aed0 msgr2=0x7f327c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f327c06aed0 0x7f327c06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3288007fd0 tx=0x7f3288009040 comp rx=0 tx=0).stop 2023-11-24T07:58:43.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 msgr2=0x7f329814a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 0x7f329814a2b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f3294004750 comp rx=0 tx=0).stop 2023-11-24T07:58:43.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 shutdown_connections 2023-11-24T07:58:43.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f327c06aed0 0x7f327c06d390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f329815df10 0x7f329815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32980ce4d0 0x7f329814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 --2- 172.21.15.32:0/2390230450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32980d1a30 0x7f329814a2b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 >> 172.21.15.32:0/2390230450 conn(0x7f329809aa00 msgr2=0x7f32980b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:43.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 shutdown_connections 2023-11-24T07:58:43.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.185+0000 7f329f9aa700 1 -- 172.21.15.32:0/2390230450 wait complete. 2023-11-24T07:58:43.192 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T07:58:43.199 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-24T07:58:42.530316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219104,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755616,"statfs":{"total":773094113280,"available":772869750784,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.896835"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210697+0000","last_change":"2023-11-24T07:58:38.673717+0000","last_active":"2023-11-24T07:58:40.210697+0000","last_peered":"2023-11-24T07:58:40.210697+0000","last_clean":"2023-11-24T07:58:40.210697+0000","last_became_active":"2023-11-24T07:58:38.673289+0000","last_became_peered":"2023-11-24T07:58:38.673289+0000","last_unstale":"2023-11-24T07:58:40.210697+0000","last_undegraded":"2023-11-24T07:58:40.210697+0000","last_fullsized":"2023-11-24T07:58:40.210697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:16:17.544102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209942+0000","last_change":"2023-11-24T07:58:38.666572+0000","last_active":"2023-11-24T07:58:40.209942+0000","last_peered":"2023-11-24T07:58:40.209942+0000","last_clean":"2023-11-24T07:58:40.209942+0000","last_became_active":"2023-11-24T07:58:38.666220+0000","last_became_peered":"2023-11-24T07:58:38.666220+0000","last_unstale":"2023-11-24T07:58:40.209942+0000","last_undegraded":"2023-11-24T07:58:40.209942+0000","last_fullsized":"2023-11-24T07:58:40.209942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670074+0000","last_change":"2023-11-24T07:58:38.664139+0000","last_active":"2023-11-24T07:58:39.670074+0000","last_peered":"2023-11-24T07:58:39.670074+0000","last_clean":"2023-11-24T07:58:39.670074+0000","last_became_active":"2023-11-24T07:58:38.660601+0000","last_became_peered":"2023-11-24T07:58:38.660601+0000","last_unstale":"2023-11-24T07:58:39.670074+0000","last_undegraded":"2023-11-24T07:58:39.670074+0000","last_fullsized":"2023-11-24T07:58:39.670074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662796+0000","last_change":"2023-11-24T07:58:38.670834+0000","last_active":"2023-11-24T07:58:39.662796+0000","last_peered":"2023-11-24T07:58:39.662796+0000","last_clean":"2023-11-24T07:58:39.662796+0000","last_became_active":"2023-11-24T07:58:38.670567+0000","last_became_peered":"2023-11-24T07:58:38.670567+0000","last_unstale":"2023-11-24T07:58:39.662796+0000","last_undegraded":"2023-11-24T07:58:39.662796+0000","last_fullsized":"2023-11-24T07:58:39.662796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210112+0000","last_change":"2023-11-24T07:58:38.657900+0000","last_active":"2023-11-24T07:58:40.210112+0000","last_peered":"2023-11-24T07:58:40.210112+0000","last_clean":"2023-11-24T07:58:40.210112+0000","last_became_active":"2023-11-24T07:58:38.657374+0000","last_became_peered":"2023-11-24T07:58:38.657374+0000","last_unstale":"2023-11-24T07:58:40.210112+0000","last_undegraded":"2023-11-24T07:58:40.210112+0000","last_fullsized":"2023-11-24T07:58:40.210112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662085+0000","last_change":"2023-11-24T07:58:38.668973+0000","last_active":"2023-11-24T07:58:39.662085+0000","last_peered":"2023-11-24T07:58:39.662085+0000","last_clean":"2023-11-24T07:58:39.662085+0000","last_became_active":"2023-11-24T07:58:38.668660+0000","last_became_peered":"2023-11-24T07:58:38.668660+0000","last_unstale":"2023-11-24T07:58:39.662085+0000","last_undegraded":"2023-11-24T07:58:39.662085+0000","last_fullsized":"2023-11-24T07:58:39.662085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209727+0000","last_change":"2023-11-24T07:58:38.670604+0000","last_active":"2023-11-24T07:58:40.209727+0000","last_peered":"2023-11-24T07:58:40.209727+0000","last_clean":"2023-11-24T07:58:40.209727+0000","last_became_active":"2023-11-24T07:58:38.670304+0000","last_became_peered":"2023-11-24T07:58:38.670304+0000","last_unstale":"2023-11-24T07:58:40.209727+0000","last_undegraded":"2023-11-24T07:58:40.209727+0000","last_fullsized":"2023-11-24T07:58:40.209727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655626+0000","last_change":"2023-11-24T07:58:38.671094+0000","last_active":"2023-11-24T07:58:39.655626+0000","last_peered":"2023-11-24T07:58:39.655626+0000","last_clean":"2023-11-24T07:58:39.655626+0000","last_became_active":"2023-11-24T07:58:38.670534+0000","last_became_peered":"2023-11-24T07:58:38.670534+0000","last_unstale":"2023-11-24T07:58:39.655626+0000","last_undegraded":"2023-11-24T07:58:39.655626+0000","last_fullsized":"2023-11-24T07:58:39.655626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210625+0000","last_change":"2023-11-24T07:58:38.673877+0000","last_active":"2023-11-24T07:58:40.210625+0000","last_peered":"2023-11-24T07:58:40.210625+0000","last_clean":"2023-11-24T07:58:40.210625+0000","last_became_active":"2023-11-24T07:58:38.673449+0000","last_became_peered":"2023-11-24T07:58:38.673449+0000","last_unstale":"2023-11-24T07:58:40.210625+0000","last_undegraded":"2023-11-24T07:58:40.210625+0000","last_fullsized":"2023-11-24T07:58:40.210625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661371+0000","last_change":"2023-11-24T07:58:38.667090+0000","last_active":"2023-11-24T07:58:39.661371+0000","last_peered":"2023-11-24T07:58:39.661371+0000","last_clean":"2023-11-24T07:58:39.661371+0000","last_became_active":"2023-11-24T07:58:38.665486+0000","last_became_peered":"2023-11-24T07:58:38.665486+0000","last_unstale":"2023-11-24T07:58:39.661371+0000","last_undegraded":"2023-11-24T07:58:39.661371+0000","last_fullsized":"2023-11-24T07:58:39.661371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661214+0000","last_change":"2023-11-24T07:58:38.666051+0000","last_active":"2023-11-24T07:58:39.661214+0000","last_peered":"2023-11-24T07:58:39.661214+0000","last_clean":"2023-11-24T07:58:39.661214+0000","last_became_active":"2023-11-24T07:58:38.665574+0000","last_became_peered":"2023-11-24T07:58:38.665574+0000","last_unstale":"2023-11-24T07:58:39.661214+0000","last_undegraded":"2023-11-24T07:58:39.661214+0000","last_fullsized":"2023-11-24T07:58:39.661214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210541+0000","last_change":"2023-11-24T07:58:38.667776+0000","last_active":"2023-11-24T07:58:40.210541+0000","last_peered":"2023-11-24T07:58:40.210541+0000","last_clean":"2023-11-24T07:58:40.210541+0000","last_became_active":"2023-11-24T07:58:38.667513+0000","last_became_peered":"2023-11-24T07:58:38.667513+0000","last_unstale":"2023-11-24T07:58:40.210541+0000","last_undegraded":"2023-11-24T07:58:40.210541+0000","last_fullsized":"2023-11-24T07:58:40.210541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655517+0000","last_change":"2023-11-24T07:58:38.672201+0000","last_active":"2023-11-24T07:58:39.655517+0000","last_peered":"2023-11-24T07:58:39.655517+0000","last_clean":"2023-11-24T07:58:39.655517+0000","last_became_active":"2023-11-24T07:58:38.671869+0000","last_became_peered":"2023-11-24T07:58:38.671869+0000","last_unstale":"2023-11-24T07:58:39.655517+0000","last_undegraded":"2023-11-24T07:58:39.655517+0000","last_fullsized":"2023-11-24T07:58:39.655517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670156+0000","last_change":"2023-11-24T07:58:38.662685+0000","last_active":"2023-11-24T07:58:39.670156+0000","last_peered":"2023-11-24T07:58:39.670156+0000","last_clean":"2023-11-24T07:58:39.670156+0000","last_became_active":"2023-11-24T07:58:38.660043+0000","last_became_peered":"2023-11-24T07:58:38.660043+0000","last_unstale":"2023-11-24T07:58:39.670156+0000","last_undegraded":"2023-11-24T07:58:39.670156+0000","last_fullsized":"2023-11-24T07:58:39.670156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209459+0000","last_change":"2023-11-24T07:58:38.667653+0000","last_active":"2023-11-24T07:58:40.209459+0000","last_peered":"2023-11-24T07:58:40.209459+0000","last_clean":"2023-11-24T07:58:40.209459+0000","last_became_active":"2023-11-24T07:58:38.667388+0000","last_became_peered":"2023-11-24T07:58:38.667388+0000","last_unstale":"2023-11-24T07:58:40.209459+0000","last_undegraded":"2023-11-24T07:58:40.209459+0000","last_fullsized":"2023-11-24T07:58:40.209459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661870+0000","last_change":"2023-11-24T07:58:38.669289+0000","last_active":"2023-11-24T07:58:39.661870+0000","last_peered":"2023-11-24T07:58:39.661870+0000","last_clean":"2023-11-24T07:58:39.661870+0000","last_became_active":"2023-11-24T07:58:38.668715+0000","last_became_peered":"2023-11-24T07:58:38.668715+0000","last_unstale":"2023-11-24T07:58:39.661870+0000","last_undegraded":"2023-11-24T07:58:39.661870+0000","last_fullsized":"2023-11-24T07:58:39.661870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662528+0000","last_change":"2023-11-24T07:58:38.674765+0000","last_active":"2023-11-24T07:58:39.662528+0000","last_peered":"2023-11-24T07:58:39.662528+0000","last_clean":"2023-11-24T07:58:39.662528+0000","last_became_active":"2023-11-24T07:58:38.674479+0000","last_became_peered":"2023-11-24T07:58:38.674479+0000","last_unstale":"2023-11-24T07:58:39.662528+0000","last_undegraded":"2023-11-24T07:58:39.662528+0000","last_fullsized":"2023-11-24T07:58:39.662528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210293+0000","last_change":"2023-11-24T07:58:38.667446+0000","last_active":"2023-11-24T07:58:40.210293+0000","last_peered":"2023-11-24T07:58:40.210293+0000","last_clean":"2023-11-24T07:58:40.210293+0000","last_became_active":"2023-11-24T07:58:38.667164+0000","last_became_peered":"2023-11-24T07:58:38.667164+0000","last_unstale":"2023-11-24T07:58:40.210293+0000","last_undegraded":"2023-11-24T07:58:40.210293+0000","last_fullsized":"2023-11-24T07:58:40.210293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210359+0000","last_change":"2023-11-24T07:58:38.663509+0000","last_active":"2023-11-24T07:58:40.210359+0000","last_peered":"2023-11-24T07:58:40.210359+0000","last_clean":"2023-11-24T07:58:40.210359+0000","last_became_active":"2023-11-24T07:58:38.661576+0000","last_became_peered":"2023-11-24T07:58:38.661576+0000","last_unstale":"2023-11-24T07:58:40.210359+0000","last_undegraded":"2023-11-24T07:58:40.210359+0000","last_fullsized":"2023-11-24T07:58:40.210359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669966+0000","last_change":"2023-11-24T07:58:37.664997+0000","last_active":"2023-11-24T07:58:39.669966+0000","last_peered":"2023-11-24T07:58:39.669966+0000","last_clean":"2023-11-24T07:58:39.669966+0000","last_became_active":"2023-11-24T07:58:37.664509+0000","last_became_peered":"2023-11-24T07:58:37.664509+0000","last_unstale":"2023-11-24T07:58:39.669966+0000","last_undegraded":"2023-11-24T07:58:39.669966+0000","last_fullsized":"2023-11-24T07:58:39.669966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656179+0000","last_change":"2023-11-24T07:58:38.671945+0000","last_active":"2023-11-24T07:58:39.656179+0000","last_peered":"2023-11-24T07:58:39.656179+0000","last_clean":"2023-11-24T07:58:39.656179+0000","last_became_active":"2023-11-24T07:58:38.671503+0000","last_became_peered":"2023-11-24T07:58:38.671503+0000","last_unstale":"2023-11-24T07:58:39.656179+0000","last_undegraded":"2023-11-24T07:58:39.656179+0000","last_fullsized":"2023-11-24T07:58:39.656179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656004+0000","last_change":"2023-11-24T07:58:37.664969+0000","last_active":"2023-11-24T07:58:39.656004+0000","last_peered":"2023-11-24T07:58:39.656004+0000","last_clean":"2023-11-24T07:58:39.656004+0000","last_became_active":"2023-11-24T07:58:37.664661+0000","last_became_peered":"2023-11-24T07:58:37.664661+0000","last_unstale":"2023-11-24T07:58:39.656004+0000","last_undegraded":"2023-11-24T07:58:39.656004+0000","last_fullsized":"2023-11-24T07:58:39.656004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670016+0000","last_change":"2023-11-24T07:58:38.670070+0000","last_active":"2023-11-24T07:58:39.670016+0000","last_peered":"2023-11-24T07:58:39.670016+0000","last_clean":"2023-11-24T07:58:39.670016+0000","last_became_active":"2023-11-24T07:58:38.669739+0000","last_became_peered":"2023-11-24T07:58:38.669739+0000","last_unstale":"2023-11-24T07:58:39.670016+0000","last_undegraded":"2023-11-24T07:58:39.670016+0000","last_fullsized":"2023-11-24T07:58:39.670016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:04.076457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210695+0000","last_change":"2023-11-24T07:58:37.663122+0000","last_active":"2023-11-24T07:58:40.210695+0000","last_peered":"2023-11-24T07:58:40.210695+0000","last_clean":"2023-11-24T07:58:40.210695+0000","last_became_active":"2023-11-24T07:58:37.662846+0000","last_became_peered":"2023-11-24T07:58:37.662846+0000","last_unstale":"2023-11-24T07:58:40.210695+0000","last_undegraded":"2023-11-24T07:58:40.210695+0000","last_fullsized":"2023-11-24T07:58:40.210695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:56.244472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662566+0000","last_change":"2023-11-24T07:58:38.669302+0000","last_active":"2023-11-24T07:58:39.662566+0000","last_peered":"2023-11-24T07:58:39.662566+0000","last_clean":"2023-11-24T07:58:39.662566+0000","last_became_active":"2023-11-24T07:58:38.669023+0000","last_became_peered":"2023-11-24T07:58:38.669023+0000","last_unstale":"2023-11-24T07:58:39.662566+0000","last_undegraded":"2023-11-24T07:58:39.662566+0000","last_fullsized":"2023-11-24T07:58:39.662566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210236+0000","last_change":"2023-11-24T07:58:37.659022+0000","last_active":"2023-11-24T07:58:40.210236+0000","last_peered":"2023-11-24T07:58:40.210236+0000","last_clean":"2023-11-24T07:58:40.210236+0000","last_became_active":"2023-11-24T07:58:37.658698+0000","last_became_peered":"2023-11-24T07:58:37.658698+0000","last_unstale":"2023-11-24T07:58:40.210236+0000","last_undegraded":"2023-11-24T07:58:40.210236+0000","last_fullsized":"2023-11-24T07:58:40.210236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662404+0000","last_change":"2023-11-24T07:58:38.669294+0000","last_active":"2023-11-24T07:58:39.662404+0000","last_peered":"2023-11-24T07:58:39.662404+0000","last_clean":"2023-11-24T07:58:39.662404+0000","last_became_active":"2023-11-24T07:58:38.668636+0000","last_became_peered":"2023-11-24T07:58:38.668636+0000","last_unstale":"2023-11-24T07:58:39.662404+0000","last_undegraded":"2023-11-24T07:58:39.662404+0000","last_fullsized":"2023-11-24T07:58:39.662404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655621+0000","last_change":"2023-11-24T07:58:37.655113+0000","last_active":"2023-11-24T07:58:39.655621+0000","last_peered":"2023-11-24T07:58:39.655621+0000","last_clean":"2023-11-24T07:58:39.655621+0000","last_became_active":"2023-11-24T07:58:37.654784+0000","last_became_peered":"2023-11-24T07:58:37.654784+0000","last_unstale":"2023-11-24T07:58:39.655621+0000","last_undegraded":"2023-11-24T07:58:39.655621+0000","last_fullsized":"2023-11-24T07:58:39.655621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209899+0000","last_change":"2023-11-24T07:58:38.653315+0000","last_active":"2023-11-24T07:58:40.209899+0000","last_peered":"2023-11-24T07:58:40.209899+0000","last_clean":"2023-11-24T07:58:40.209899+0000","last_became_active":"2023-11-24T07:58:38.653027+0000","last_became_peered":"2023-11-24T07:58:38.653027+0000","last_unstale":"2023-11-24T07:58:40.209899+0000","last_undegraded":"2023-11-24T07:58:40.209899+0000","last_fullsized":"2023-11-24T07:58:40.209899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:03.244592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660984+0000","last_change":"2023-11-24T07:58:37.660782+0000","last_active":"2023-11-24T07:58:39.660984+0000","last_peered":"2023-11-24T07:58:39.660984+0000","last_clean":"2023-11-24T07:58:39.660984+0000","last_became_active":"2023-11-24T07:58:37.660419+0000","last_became_peered":"2023-11-24T07:58:37.660419+0000","last_unstale":"2023-11-24T07:58:39.660984+0000","last_undegraded":"2023-11-24T07:58:39.660984+0000","last_fullsized":"2023-11-24T07:58:39.660984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670254+0000","last_change":"2023-11-24T07:58:38.667006+0000","last_active":"2023-11-24T07:58:39.670254+0000","last_peered":"2023-11-24T07:58:39.670254+0000","last_clean":"2023-11-24T07:58:39.670254+0000","last_became_active":"2023-11-24T07:58:38.665254+0000","last_became_peered":"2023-11-24T07:58:38.665254+0000","last_unstale":"2023-11-24T07:58:39.670254+0000","last_undegraded":"2023-11-24T07:58:39.670254+0000","last_fullsized":"2023-11-24T07:58:39.670254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655425+0000","last_change":"2023-11-24T07:58:37.660588+0000","last_active":"2023-11-24T07:58:39.655425+0000","last_peered":"2023-11-24T07:58:39.655425+0000","last_clean":"2023-11-24T07:58:39.655425+0000","last_became_active":"2023-11-24T07:58:37.660295+0000","last_became_peered":"2023-11-24T07:58:37.660295+0000","last_unstale":"2023-11-24T07:58:39.655425+0000","last_undegraded":"2023-11-24T07:58:39.655425+0000","last_fullsized":"2023-11-24T07:58:39.655425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655528+0000","last_change":"2023-11-24T07:58:38.671305+0000","last_active":"2023-11-24T07:58:39.655528+0000","last_peered":"2023-11-24T07:58:39.655528+0000","last_clean":"2023-11-24T07:58:39.655528+0000","last_became_active":"2023-11-24T07:58:38.670974+0000","last_became_peered":"2023-11-24T07:58:38.670974+0000","last_unstale":"2023-11-24T07:58:39.655528+0000","last_undegraded":"2023-11-24T07:58:39.655528+0000","last_fullsized":"2023-11-24T07:58:39.655528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661837+0000","last_change":"2023-11-24T07:58:37.658192+0000","last_active":"2023-11-24T07:58:39.661837+0000","last_peered":"2023-11-24T07:58:39.661837+0000","last_clean":"2023-11-24T07:58:39.661837+0000","last_became_active":"2023-11-24T07:58:37.657283+0000","last_became_peered":"2023-11-24T07:58:37.657283+0000","last_unstale":"2023-11-24T07:58:39.661837+0000","last_undegraded":"2023-11-24T07:58:39.661837+0000","last_fullsized":"2023-11-24T07:58:39.661837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209719+0000","last_change":"2023-11-24T07:58:38.655330+0000","last_active":"2023-11-24T07:58:40.209719+0000","last_peered":"2023-11-24T07:58:40.209719+0000","last_clean":"2023-11-24T07:58:40.209719+0000","last_became_active":"2023-11-24T07:58:38.654988+0000","last_became_peered":"2023-11-24T07:58:38.654988+0000","last_unstale":"2023-11-24T07:58:40.209719+0000","last_undegraded":"2023-11-24T07:58:40.209719+0000","last_fullsized":"2023-11-24T07:58:40.209719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661550+0000","last_change":"2023-11-24T07:58:37.663203+0000","last_active":"2023-11-24T07:58:39.661550+0000","last_peered":"2023-11-24T07:58:39.661550+0000","last_clean":"2023-11-24T07:58:39.661550+0000","last_became_active":"2023-11-24T07:58:37.662842+0000","last_became_peered":"2023-11-24T07:58:37.662842+0000","last_unstale":"2023-11-24T07:58:39.661550+0000","last_undegraded":"2023-11-24T07:58:39.661550+0000","last_fullsized":"2023-11-24T07:58:39.661550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662636+0000","last_change":"2023-11-24T07:58:38.662184+0000","last_active":"2023-11-24T07:58:39.662636+0000","last_peered":"2023-11-24T07:58:39.662636+0000","last_clean":"2023-11-24T07:58:39.662636+0000","last_became_active":"2023-11-24T07:58:38.661870+0000","last_became_peered":"2023-11-24T07:58:38.661870+0000","last_unstale":"2023-11-24T07:58:39.662636+0000","last_undegraded":"2023-11-24T07:58:39.662636+0000","last_fullsized":"2023-11-24T07:58:39.662636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662519+0000","last_change":"2023-11-24T07:58:37.663425+0000","last_active":"2023-11-24T07:58:39.662519+0000","last_peered":"2023-11-24T07:58:39.662519+0000","last_clean":"2023-11-24T07:58:39.662519+0000","last_became_active":"2023-11-24T07:58:37.663165+0000","last_became_peered":"2023-11-24T07:58:37.663165+0000","last_unstale":"2023-11-24T07:58:39.662519+0000","last_undegraded":"2023-11-24T07:58:39.662519+0000","last_fullsized":"2023-11-24T07:58:39.662519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670128+0000","last_change":"2023-11-24T07:58:38.669831+0000","last_active":"2023-11-24T07:58:39.670128+0000","last_peered":"2023-11-24T07:58:39.670128+0000","last_clean":"2023-11-24T07:58:39.670128+0000","last_became_active":"2023-11-24T07:58:38.669524+0000","last_became_peered":"2023-11-24T07:58:38.669524+0000","last_unstale":"2023-11-24T07:58:39.670128+0000","last_undegraded":"2023-11-24T07:58:39.670128+0000","last_fullsized":"2023-11-24T07:58:39.670128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209893+0000","last_change":"2023-11-24T07:58:37.656248+0000","last_active":"2023-11-24T07:58:40.209893+0000","last_peered":"2023-11-24T07:58:40.209893+0000","last_clean":"2023-11-24T07:58:40.209893+0000","last_became_active":"2023-11-24T07:58:37.655538+0000","last_became_peered":"2023-11-24T07:58:37.655538+0000","last_unstale":"2023-11-24T07:58:40.209893+0000","last_undegraded":"2023-11-24T07:58:40.209893+0000","last_fullsized":"2023-11-24T07:58:40.209893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210389+0000","last_change":"2023-11-24T07:58:38.660386+0000","last_active":"2023-11-24T07:58:40.210389+0000","last_peered":"2023-11-24T07:58:40.210389+0000","last_clean":"2023-11-24T07:58:40.210389+0000","last_became_active":"2023-11-24T07:58:38.659454+0000","last_became_peered":"2023-11-24T07:58:38.659454+0000","last_unstale":"2023-11-24T07:58:40.210389+0000","last_undegraded":"2023-11-24T07:58:40.210389+0000","last_fullsized":"2023-11-24T07:58:40.210389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661142+0000","last_change":"2023-11-24T07:58:37.663115+0000","last_active":"2023-11-24T07:58:39.661142+0000","last_peered":"2023-11-24T07:58:39.661142+0000","last_clean":"2023-11-24T07:58:39.661142+0000","last_became_active":"2023-11-24T07:58:37.662759+0000","last_became_peered":"2023-11-24T07:58:37.662759+0000","last_unstale":"2023-11-24T07:58:39.661142+0000","last_undegraded":"2023-11-24T07:58:39.661142+0000","last_fullsized":"2023-11-24T07:58:39.661142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210514+0000","last_change":"2023-11-24T07:58:38.677184+0000","last_active":"2023-11-24T07:58:40.210514+0000","last_peered":"2023-11-24T07:58:40.210514+0000","last_clean":"2023-11-24T07:58:40.210514+0000","last_became_active":"2023-11-24T07:58:38.676729+0000","last_became_peered":"2023-11-24T07:58:38.676729+0000","last_unstale":"2023-11-24T07:58:40.210514+0000","last_undegraded":"2023-11-24T07:58:40.210514+0000","last_fullsized":"2023-11-24T07:58:40.210514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670164+0000","last_change":"2023-11-24T07:58:37.659853+0000","last_active":"2023-11-24T07:58:39.670164+0000","last_peered":"2023-11-24T07:58:39.670164+0000","last_clean":"2023-11-24T07:58:39.670164+0000","last_became_active":"2023-11-24T07:58:37.659563+0000","last_became_peered":"2023-11-24T07:58:37.659563+0000","last_unstale":"2023-11-24T07:58:39.670164+0000","last_undegraded":"2023-11-24T07:58:39.670164+0000","last_fullsized":"2023-11-24T07:58:39.670164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670259+0000","last_change":"2023-11-24T07:58:38.668668+0000","last_active":"2023-11-24T07:58:39.670259+0000","last_peered":"2023-11-24T07:58:39.670259+0000","last_clean":"2023-11-24T07:58:39.670259+0000","last_became_active":"2023-11-24T07:58:38.668181+0000","last_became_peered":"2023-11-24T07:58:38.668181+0000","last_unstale":"2023-11-24T07:58:39.670259+0000","last_undegraded":"2023-11-24T07:58:39.670259+0000","last_fullsized":"2023-11-24T07:58:39.670259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662643+0000","last_change":"2023-11-24T07:58:37.656567+0000","last_active":"2023-11-24T07:58:39.662643+0000","last_peered":"2023-11-24T07:58:39.662643+0000","last_clean":"2023-11-24T07:58:39.662643+0000","last_became_active":"2023-11-24T07:58:37.656024+0000","last_became_peered":"2023-11-24T07:58:37.656024+0000","last_unstale":"2023-11-24T07:58:39.662643+0000","last_undegraded":"2023-11-24T07:58:39.662643+0000","last_fullsized":"2023-11-24T07:58:39.662643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210630+0000","last_change":"2023-11-24T07:58:38.664336+0000","last_active":"2023-11-24T07:58:40.210630+0000","last_peered":"2023-11-24T07:58:40.210630+0000","last_clean":"2023-11-24T07:58:40.210630+0000","last_became_active":"2023-11-24T07:58:38.664080+0000","last_became_peered":"2023-11-24T07:58:38.664080+0000","last_unstale":"2023-11-24T07:58:40.210630+0000","last_undegraded":"2023-11-24T07:58:40.210630+0000","last_fullsized":"2023-11-24T07:58:40.210630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660834+0000","last_change":"2023-11-24T07:58:37.660791+0000","last_active":"2023-11-24T07:58:39.660834+0000","last_peered":"2023-11-24T07:58:39.660834+0000","last_clean":"2023-11-24T07:58:39.660834+0000","last_became_active":"2023-11-24T07:58:37.660488+0000","last_became_peered":"2023-11-24T07:58:37.660488+0000","last_unstale":"2023-11-24T07:58:39.660834+0000","last_undegraded":"2023-11-24T07:58:39.660834+0000","last_fullsized":"2023-11-24T07:58:39.660834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660799+0000","last_change":"2023-11-24T07:58:38.670298+0000","last_active":"2023-11-24T07:58:39.660799+0000","last_peered":"2023-11-24T07:58:39.660799+0000","last_clean":"2023-11-24T07:58:39.660799+0000","last_became_active":"2023-11-24T07:58:38.669956+0000","last_became_peered":"2023-11-24T07:58:38.669956+0000","last_unstale":"2023-11-24T07:58:39.660799+0000","last_undegraded":"2023-11-24T07:58:39.660799+0000","last_fullsized":"2023-11-24T07:58:39.660799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209457+0000","last_change":"2023-11-24T07:58:37.659643+0000","last_active":"2023-11-24T07:58:40.209457+0000","last_peered":"2023-11-24T07:58:40.209457+0000","last_clean":"2023-11-24T07:58:40.209457+0000","last_became_active":"2023-11-24T07:58:37.659317+0000","last_became_peered":"2023-11-24T07:58:37.659317+0000","last_unstale":"2023-11-24T07:58:40.209457+0000","last_undegraded":"2023-11-24T07:58:40.209457+0000","last_fullsized":"2023-11-24T07:58:40.209457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:32.387984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209982+0000","last_change":"2023-11-24T07:58:38.667271+0000","last_active":"2023-11-24T07:58:40.209982+0000","last_peered":"2023-11-24T07:58:40.209982+0000","last_clean":"2023-11-24T07:58:40.209982+0000","last_became_active":"2023-11-24T07:58:38.666904+0000","last_became_peered":"2023-11-24T07:58:38.666904+0000","last_unstale":"2023-11-24T07:58:40.209982+0000","last_undegraded":"2023-11-24T07:58:40.209982+0000","last_fullsized":"2023-11-24T07:58:40.209982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210573+0000","last_change":"2023-11-24T07:58:37.657085+0000","last_active":"2023-11-24T07:58:40.210573+0000","last_peered":"2023-11-24T07:58:40.210573+0000","last_clean":"2023-11-24T07:58:40.210573+0000","last_became_active":"2023-11-24T07:58:37.656676+0000","last_became_peered":"2023-11-24T07:58:37.656676+0000","last_unstale":"2023-11-24T07:58:40.210573+0000","last_undegraded":"2023-11-24T07:58:40.210573+0000","last_fullsized":"2023-11-24T07:58:40.210573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:45.701317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210306+0000","last_change":"2023-11-24T07:58:38.664417+0000","last_active":"2023-11-24T07:58:40.210306+0000","last_peered":"2023-11-24T07:58:40.210306+0000","last_clean":"2023-11-24T07:58:40.210306+0000","last_became_active":"2023-11-24T07:58:38.664161+0000","last_became_peered":"2023-11-24T07:58:38.664161+0000","last_unstale":"2023-11-24T07:58:40.210306+0000","last_undegraded":"2023-11-24T07:58:40.210306+0000","last_fullsized":"2023-11-24T07:58:40.210306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209807+0000","last_change":"2023-11-24T07:58:37.660866+0000","last_active":"2023-11-24T07:58:40.209807+0000","last_peered":"2023-11-24T07:58:40.209807+0000","last_clean":"2023-11-24T07:58:40.209807+0000","last_became_active":"2023-11-24T07:58:37.660577+0000","last_became_peered":"2023-11-24T07:58:37.660577+0000","last_unstale":"2023-11-24T07:58:40.209807+0000","last_undegraded":"2023-11-24T07:58:40.209807+0000","last_fullsized":"2023-11-24T07:58:40.209807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662441+0000","last_change":"2023-11-24T07:58:38.674685+0000","last_active":"2023-11-24T07:58:39.662441+0000","last_peered":"2023-11-24T07:58:39.662441+0000","last_clean":"2023-11-24T07:58:39.662441+0000","last_became_active":"2023-11-24T07:58:38.674366+0000","last_became_peered":"2023-11-24T07:58:38.674366+0000","last_unstale":"2023-11-24T07:58:39.662441+0000","last_undegraded":"2023-11-24T07:58:39.662441+0000","last_fullsized":"2023-11-24T07:58:39.662441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662580+0000","last_change":"2023-11-24T07:58:37.655735+0000","last_active":"2023-11-24T07:58:39.662580+0000","last_peered":"2023-11-24T07:58:39.662580+0000","last_clean":"2023-11-24T07:58:39.662580+0000","last_became_active":"2023-11-24T07:58:37.655444+0000","last_became_peered":"2023-11-24T07:58:37.655444+0000","last_unstale":"2023-11-24T07:58:39.662580+0000","last_undegraded":"2023-11-24T07:58:39.662580+0000","last_fullsized":"2023-11-24T07:58:39.662580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:10.569901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210250+0000","last_change":"2023-11-24T07:58:38.667440+0000","last_active":"2023-11-24T07:58:40.210250+0000","last_peered":"2023-11-24T07:58:40.210250+0000","last_clean":"2023-11-24T07:58:40.210250+0000","last_became_active":"2023-11-24T07:58:38.666914+0000","last_became_peered":"2023-11-24T07:58:38.666914+0000","last_unstale":"2023-11-24T07:58:40.210250+0000","last_undegraded":"2023-11-24T07:58:40.210250+0000","last_fullsized":"2023-11-24T07:58:40.210250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:19.761593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209468+0000","last_change":"2023-11-24T07:58:38.655018+0000","last_active":"2023-11-24T07:58:40.209468+0000","last_peered":"2023-11-24T07:58:40.209468+0000","last_clean":"2023-11-24T07:58:40.209468+0000","last_became_active":"2023-11-24T07:58:38.654604+0000","last_became_peered":"2023-11-24T07:58:38.654604+0000","last_unstale":"2023-11-24T07:58:40.209468+0000","last_undegraded":"2023-11-24T07:58:40.209468+0000","last_fullsized":"2023-11-24T07:58:40.209468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209541+0000","last_change":"2023-11-24T07:58:37.660113+0000","last_active":"2023-11-24T07:58:40.209541+0000","last_peered":"2023-11-24T07:58:40.209541+0000","last_clean":"2023-11-24T07:58:40.209541+0000","last_became_active":"2023-11-24T07:58:37.659747+0000","last_became_peered":"2023-11-24T07:58:37.659747+0000","last_unstale":"2023-11-24T07:58:40.209541+0000","last_undegraded":"2023-11-24T07:58:40.209541+0000","last_fullsized":"2023-11-24T07:58:40.209541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:02.242362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655662+0000","last_change":"2023-11-24T07:58:38.670798+0000","last_active":"2023-11-24T07:58:39.655662+0000","last_peered":"2023-11-24T07:58:39.655662+0000","last_clean":"2023-11-24T07:58:39.655662+0000","last_became_active":"2023-11-24T07:58:38.670464+0000","last_became_peered":"2023-11-24T07:58:38.670464+0000","last_unstale":"2023-11-24T07:58:39.655662+0000","last_undegraded":"2023-11-24T07:58:39.655662+0000","last_fullsized":"2023-11-24T07:58:39.655662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662521+0000","last_change":"2023-11-24T07:58:37.661568+0000","last_active":"2023-11-24T07:58:39.662521+0000","last_peered":"2023-11-24T07:58:39.662521+0000","last_clean":"2023-11-24T07:58:39.662521+0000","last_became_active":"2023-11-24T07:58:37.661211+0000","last_became_peered":"2023-11-24T07:58:37.661211+0000","last_unstale":"2023-11-24T07:58:39.662521+0000","last_undegraded":"2023-11-24T07:58:39.662521+0000","last_fullsized":"2023-11-24T07:58:39.662521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:58:38.650810+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:58:38.650810+0000","last_peered":"2023-11-24T07:58:38.650810+0000","last_clean":"2023-11-24T07:58:38.650810+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:58:38.650810+0000","last_undegraded":"2023-11-24T07:58:38.650810+0000","last_fullsized":"2023-11-24T07:58:38.650810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669820+0000","last_change":"2023-11-24T07:58:38.667012+0000","last_active":"2023-11-24T07:58:39.669820+0000","last_peered":"2023-11-24T07:58:39.669820+0000","last_clean":"2023-11-24T07:58:39.669820+0000","last_became_active":"2023-11-24T07:58:38.666604+0000","last_became_peered":"2023-11-24T07:58:38.666604+0000","last_unstale":"2023-11-24T07:58:39.669820+0000","last_undegraded":"2023-11-24T07:58:39.669820+0000","last_fullsized":"2023-11-24T07:58:39.669820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.085407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209908+0000","last_change":"2023-11-24T07:58:37.655598+0000","last_active":"2023-11-24T07:58:40.209908+0000","last_peered":"2023-11-24T07:58:40.209908+0000","last_clean":"2023-11-24T07:58:40.209908+0000","last_became_active":"2023-11-24T07:58:37.655321+0000","last_became_peered":"2023-11-24T07:58:37.655321+0000","last_unstale":"2023-11-24T07:58:40.209908+0000","last_undegraded":"2023-11-24T07:58:40.209908+0000","last_fullsized":"2023-11-24T07:58:40.209908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210609+0000","last_change":"2023-11-24T07:58:37.659615+0000","last_active":"2023-11-24T07:58:40.210609+0000","last_peered":"2023-11-24T07:58:40.210609+0000","last_clean":"2023-11-24T07:58:40.210609+0000","last_became_active":"2023-11-24T07:58:37.659362+0000","last_became_peered":"2023-11-24T07:58:37.659362+0000","last_unstale":"2023-11-24T07:58:40.210609+0000","last_undegraded":"2023-11-24T07:58:40.210609+0000","last_fullsized":"2023-11-24T07:58:40.210609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209722+0000","last_change":"2023-11-24T07:58:38.655615+0000","last_active":"2023-11-24T07:58:40.209722+0000","last_peered":"2023-11-24T07:58:40.209722+0000","last_clean":"2023-11-24T07:58:40.209722+0000","last_became_active":"2023-11-24T07:58:38.654516+0000","last_became_peered":"2023-11-24T07:58:38.654516+0000","last_unstale":"2023-11-24T07:58:40.209722+0000","last_undegraded":"2023-11-24T07:58:40.209722+0000","last_fullsized":"2023-11-24T07:58:40.209722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209797+0000","last_change":"2023-11-24T07:58:37.663218+0000","last_active":"2023-11-24T07:58:40.209797+0000","last_peered":"2023-11-24T07:58:40.209797+0000","last_clean":"2023-11-24T07:58:40.209797+0000","last_became_active":"2023-11-24T07:58:37.662867+0000","last_became_peered":"2023-11-24T07:58:37.662867+0000","last_unstale":"2023-11-24T07:58:40.209797+0000","last_undegraded":"2023-11-24T07:58:40.209797+0000","last_fullsized":"2023-11-24T07:58:40.209797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656341+0000","last_change":"2023-11-24T07:58:38.670569+0000","last_active":"2023-11-24T07:58:39.656341+0000","last_peered":"2023-11-24T07:58:39.656341+0000","last_clean":"2023-11-24T07:58:39.656341+0000","last_became_active":"2023-11-24T07:58:38.670239+0000","last_became_peered":"2023-11-24T07:58:38.670239+0000","last_unstale":"2023-11-24T07:58:39.656341+0000","last_undegraded":"2023-11-24T07:58:39.656341+0000","last_fullsized":"2023-11-24T07:58:39.656341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661516+0000","last_change":"2023-11-24T07:58:37.667212+0000","last_active":"2023-11-24T07:58:39.661516+0000","last_peered":"2023-11-24T07:58:39.661516+0000","last_clean":"2023-11-24T07:58:39.661516+0000","last_became_active":"2023-11-24T07:58:37.666944+0000","last_became_peered":"2023-11-24T07:58:37.666944+0000","last_unstale":"2023-11-24T07:58:39.661516+0000","last_undegraded":"2023-11-24T07:58:39.661516+0000","last_fullsized":"2023-11-24T07:58:39.661516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670071+0000","last_change":"2023-11-24T07:58:38.660314+0000","last_active":"2023-11-24T07:58:39.670071+0000","last_peered":"2023-11-24T07:58:39.670071+0000","last_clean":"2023-11-24T07:58:39.670071+0000","last_became_active":"2023-11-24T07:58:38.660005+0000","last_became_peered":"2023-11-24T07:58:38.660005+0000","last_unstale":"2023-11-24T07:58:39.670071+0000","last_undegraded":"2023-11-24T07:58:39.670071+0000","last_fullsized":"2023-11-24T07:58:39.670071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209750+0000","last_change":"2023-11-24T07:58:37.667045+0000","last_active":"2023-11-24T07:58:40.209750+0000","last_peered":"2023-11-24T07:58:40.209750+0000","last_clean":"2023-11-24T07:58:40.209750+0000","last_became_active":"2023-11-24T07:58:37.666766+0000","last_became_peered":"2023-11-24T07:58:37.666766+0000","last_unstale":"2023-11-24T07:58:40.209750+0000","last_undegraded":"2023-11-24T07:58:40.209750+0000","last_fullsized":"2023-11-24T07:58:40.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209814+0000","last_change":"2023-11-24T07:58:38.652046+0000","last_active":"2023-11-24T07:58:40.209814+0000","last_peered":"2023-11-24T07:58:40.209814+0000","last_clean":"2023-11-24T07:58:40.209814+0000","last_became_active":"2023-11-24T07:58:38.651762+0000","last_became_peered":"2023-11-24T07:58:38.651762+0000","last_unstale":"2023-11-24T07:58:40.209814+0000","last_undegraded":"2023-11-24T07:58:40.209814+0000","last_fullsized":"2023-11-24T07:58:40.209814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:28.237034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209556+0000","last_change":"2023-11-24T07:58:37.655838+0000","last_active":"2023-11-24T07:58:40.209556+0000","last_peered":"2023-11-24T07:58:40.209556+0000","last_clean":"2023-11-24T07:58:40.209556+0000","last_became_active":"2023-11-24T07:58:37.655553+0000","last_became_peered":"2023-11-24T07:58:37.655553+0000","last_unstale":"2023-11-24T07:58:40.209556+0000","last_undegraded":"2023-11-24T07:58:40.209556+0000","last_fullsized":"2023-11-24T07:58:40.209556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655413+0000","last_change":"2023-11-24T07:58:38.655539+0000","last_active":"2023-11-24T07:58:39.655413+0000","last_peered":"2023-11-24T07:58:39.655413+0000","last_clean":"2023-11-24T07:58:39.655413+0000","last_became_active":"2023-11-24T07:58:38.655169+0000","last_became_peered":"2023-11-24T07:58:38.655169+0000","last_unstale":"2023-11-24T07:58:39.655413+0000","last_undegraded":"2023-11-24T07:58:39.655413+0000","last_fullsized":"2023-11-24T07:58:39.655413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209712+0000","last_change":"2023-11-24T07:58:37.655841+0000","last_active":"2023-11-24T07:58:40.209712+0000","last_peered":"2023-11-24T07:58:40.209712+0000","last_clean":"2023-11-24T07:58:40.209712+0000","last_became_active":"2023-11-24T07:58:37.655487+0000","last_became_peered":"2023-11-24T07:58:37.655487+0000","last_unstale":"2023-11-24T07:58:40.209712+0000","last_undegraded":"2023-11-24T07:58:40.209712+0000","last_fullsized":"2023-11-24T07:58:40.209712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655699+0000","last_change":"2023-11-24T07:58:38.671051+0000","last_active":"2023-11-24T07:58:39.655699+0000","last_peered":"2023-11-24T07:58:39.655699+0000","last_clean":"2023-11-24T07:58:39.655699+0000","last_became_active":"2023-11-24T07:58:38.670571+0000","last_became_peered":"2023-11-24T07:58:38.670571+0000","last_unstale":"2023-11-24T07:58:39.655699+0000","last_undegraded":"2023-11-24T07:58:39.655699+0000","last_fullsized":"2023-11-24T07:58:39.655699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210427+0000","last_change":"2023-11-24T07:58:37.655295+0000","last_active":"2023-11-24T07:58:40.210427+0000","last_peered":"2023-11-24T07:58:40.210427+0000","last_clean":"2023-11-24T07:58:40.210427+0000","last_became_active":"2023-11-24T07:58:37.654767+0000","last_became_peered":"2023-11-24T07:58:37.654767+0000","last_unstale":"2023-11-24T07:58:40.210427+0000","last_undegraded":"2023-11-24T07:58:40.210427+0000","last_fullsized":"2023-11-24T07:58:40.210427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660877+0000","last_change":"2023-11-24T07:58:38.662792+0000","last_active":"2023-11-24T07:58:39.660877+0000","last_peered":"2023-11-24T07:58:39.660877+0000","last_clean":"2023-11-24T07:58:39.660877+0000","last_became_active":"2023-11-24T07:58:38.662492+0000","last_became_peered":"2023-11-24T07:58:38.662492+0000","last_unstale":"2023-11-24T07:58:39.660877+0000","last_undegraded":"2023-11-24T07:58:39.660877+0000","last_fullsized":"2023-11-24T07:58:39.660877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210083+0000","last_change":"2023-11-24T07:58:37.660634+0000","last_active":"2023-11-24T07:58:40.210083+0000","last_peered":"2023-11-24T07:58:40.210083+0000","last_clean":"2023-11-24T07:58:40.210083+0000","last_became_active":"2023-11-24T07:58:37.660340+0000","last_became_peered":"2023-11-24T07:58:37.660340+0000","last_unstale":"2023-11-24T07:58:40.210083+0000","last_undegraded":"2023-11-24T07:58:40.210083+0000","last_fullsized":"2023-11-24T07:58:40.210083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:04.555457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210612+0000","last_change":"2023-11-24T07:58:38.672774+0000","last_active":"2023-11-24T07:58:40.210612+0000","last_peered":"2023-11-24T07:58:40.210612+0000","last_clean":"2023-11-24T07:58:40.210612+0000","last_became_active":"2023-11-24T07:58:38.672498+0000","last_became_peered":"2023-11-24T07:58:38.672498+0000","last_unstale":"2023-11-24T07:58:40.210612+0000","last_undegraded":"2023-11-24T07:58:40.210612+0000","last_fullsized":"2023-11-24T07:58:40.210612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209839+0000","last_change":"2023-11-24T07:58:37.653547+0000","last_active":"2023-11-24T07:58:40.209839+0000","last_peered":"2023-11-24T07:58:40.209839+0000","last_clean":"2023-11-24T07:58:40.209839+0000","last_became_active":"2023-11-24T07:58:37.653148+0000","last_became_peered":"2023-11-24T07:58:37.653148+0000","last_unstale":"2023-11-24T07:58:40.209839+0000","last_undegraded":"2023-11-24T07:58:40.209839+0000","last_fullsized":"2023-11-24T07:58:40.209839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662278+0000","last_change":"2023-11-24T07:58:38.676919+0000","last_active":"2023-11-24T07:58:39.662278+0000","last_peered":"2023-11-24T07:58:39.662278+0000","last_clean":"2023-11-24T07:58:39.662278+0000","last_became_active":"2023-11-24T07:58:38.676622+0000","last_became_peered":"2023-11-24T07:58:38.676622+0000","last_unstale":"2023-11-24T07:58:39.662278+0000","last_undegraded":"2023-11-24T07:58:39.662278+0000","last_fullsized":"2023-11-24T07:58:39.662278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210143+0000","last_change":"2023-11-24T07:58:37.659572+0000","last_active":"2023-11-24T07:58:40.210143+0000","last_peered":"2023-11-24T07:58:40.210143+0000","last_clean":"2023-11-24T07:58:40.210143+0000","last_became_active":"2023-11-24T07:58:37.659273+0000","last_became_peered":"2023-11-24T07:58:37.659273+0000","last_unstale":"2023-11-24T07:58:40.210143+0000","last_undegraded":"2023-11-24T07:58:40.210143+0000","last_fullsized":"2023-11-24T07:58:40.210143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210569+0000","last_change":"2023-11-24T07:58:38.673691+0000","last_active":"2023-11-24T07:58:40.210569+0000","last_peered":"2023-11-24T07:58:40.210569+0000","last_clean":"2023-11-24T07:58:40.210569+0000","last_became_active":"2023-11-24T07:58:38.673092+0000","last_became_peered":"2023-11-24T07:58:38.673092+0000","last_unstale":"2023-11-24T07:58:40.210569+0000","last_undegraded":"2023-11-24T07:58:40.210569+0000","last_fullsized":"2023-11-24T07:58:40.210569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661147+0000","last_change":"2023-11-24T07:58:38.670271+0000","last_active":"2023-11-24T07:58:39.661147+0000","last_peered":"2023-11-24T07:58:39.661147+0000","last_clean":"2023-11-24T07:58:39.661147+0000","last_became_active":"2023-11-24T07:58:38.669965+0000","last_became_peered":"2023-11-24T07:58:38.669965+0000","last_unstale":"2023-11-24T07:58:39.661147+0000","last_undegraded":"2023-11-24T07:58:39.661147+0000","last_fullsized":"2023-11-24T07:58:39.661147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209683+0000","last_change":"2023-11-24T07:58:38.663275+0000","last_active":"2023-11-24T07:58:40.209683+0000","last_peered":"2023-11-24T07:58:40.209683+0000","last_clean":"2023-11-24T07:58:40.209683+0000","last_became_active":"2023-11-24T07:58:38.662928+0000","last_became_peered":"2023-11-24T07:58:38.662928+0000","last_unstale":"2023-11-24T07:58:40.209683+0000","last_undegraded":"2023-11-24T07:58:40.209683+0000","last_fullsized":"2023-11-24T07:58:40.209683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662492+0000","last_change":"2023-11-24T07:58:38.674085+0000","last_active":"2023-11-24T07:58:39.662492+0000","last_peered":"2023-11-24T07:58:39.662492+0000","last_clean":"2023-11-24T07:58:39.662492+0000","last_became_active":"2023-11-24T07:58:38.673780+0000","last_became_peered":"2023-11-24T07:58:38.673780+0000","last_unstale":"2023-11-24T07:58:39.662492+0000","last_undegraded":"2023-11-24T07:58:39.662492+0000","last_fullsized":"2023-11-24T07:58:39.662492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670095+0000","last_change":"2023-11-24T07:58:38.676742+0000","last_active":"2023-11-24T07:58:39.670095+0000","last_peered":"2023-11-24T07:58:39.670095+0000","last_clean":"2023-11-24T07:58:39.670095+0000","last_became_active":"2023-11-24T07:58:38.676422+0000","last_became_peered":"2023-11-24T07:58:38.676422+0000","last_unstale":"2023-11-24T07:58:39.670095+0000","last_undegraded":"2023-11-24T07:58:39.670095+0000","last_fullsized":"2023-11-24T07:58:39.670095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655920+0000","last_change":"2023-11-24T07:58:38.674946+0000","last_active":"2023-11-24T07:58:39.655920+0000","last_peered":"2023-11-24T07:58:39.655920+0000","last_clean":"2023-11-24T07:58:39.655920+0000","last_became_active":"2023-11-24T07:58:38.674458+0000","last_became_peered":"2023-11-24T07:58:38.674458+0000","last_unstale":"2023-11-24T07:58:39.655920+0000","last_undegraded":"2023-11-24T07:58:39.655920+0000","last_fullsized":"2023-11-24T07:58:39.655920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209765+0000","last_change":"2023-11-24T07:58:38.656486+0000","last_active":"2023-11-24T07:58:40.209765+0000","last_peered":"2023-11-24T07:58:40.209765+0000","last_clean":"2023-11-24T07:58:40.209765+0000","last_became_active":"2023-11-24T07:58:38.654844+0000","last_became_peered":"2023-11-24T07:58:38.654844+0000","last_unstale":"2023-11-24T07:58:40.209765+0000","last_undegraded":"2023-11-24T07:58:40.209765+0000","last_fullsized":"2023-11-24T07:58:40.209765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661231+0000","last_change":"2023-11-24T07:58:38.666517+0000","last_active":"2023-11-24T07:58:39.661231+0000","last_peered":"2023-11-24T07:58:39.661231+0000","last_clean":"2023-11-24T07:58:39.661231+0000","last_became_active":"2023-11-24T07:58:38.665821+0000","last_became_peered":"2023-11-24T07:58:38.665821+0000","last_unstale":"2023-11-24T07:58:39.661231+0000","last_undegraded":"2023-11-24T07:58:39.661231+0000","last_fullsized":"2023-11-24T07:58:39.661231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656201+0000","last_change":"2023-11-24T07:58:38.672413+0000","last_active":"2023-11-24T07:58:39.656201+0000","last_peered":"2023-11-24T07:58:39.656201+0000","last_clean":"2023-11-24T07:58:39.656201+0000","last_became_active":"2023-11-24T07:58:38.672131+0000","last_became_peered":"2023-11-24T07:58:38.672131+0000","last_unstale":"2023-11-24T07:58:39.656201+0000","last_undegraded":"2023-11-24T07:58:39.656201+0000","last_fullsized":"2023-11-24T07:58:39.656201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662001+0000","last_change":"2023-11-24T07:58:38.672368+0000","last_active":"2023-11-24T07:58:39.662001+0000","last_peered":"2023-11-24T07:58:39.662001+0000","last_clean":"2023-11-24T07:58:39.662001+0000","last_became_active":"2023-11-24T07:58:38.672038+0000","last_became_peered":"2023-11-24T07:58:38.672038+0000","last_unstale":"2023-11-24T07:58:39.662001+0000","last_undegraded":"2023-11-24T07:58:39.662001+0000","last_fullsized":"2023-11-24T07:58:39.662001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655743+0000","last_change":"2023-11-24T07:58:38.671825+0000","last_active":"2023-11-24T07:58:39.655743+0000","last_peered":"2023-11-24T07:58:39.655743+0000","last_clean":"2023-11-24T07:58:39.655743+0000","last_became_active":"2023-11-24T07:58:38.671530+0000","last_became_peered":"2023-11-24T07:58:38.671530+0000","last_unstale":"2023-11-24T07:58:39.655743+0000","last_undegraded":"2023-11-24T07:58:39.655743+0000","last_fullsized":"2023-11-24T07:58:39.655743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662135+0000","last_change":"2023-11-24T07:58:38.663149+0000","last_active":"2023-11-24T07:58:39.662135+0000","last_peered":"2023-11-24T07:58:39.662135+0000","last_clean":"2023-11-24T07:58:39.662135+0000","last_became_active":"2023-11-24T07:58:38.662826+0000","last_became_peered":"2023-11-24T07:58:38.662826+0000","last_unstale":"2023-11-24T07:58:39.662135+0000","last_undegraded":"2023-11-24T07:58:39.662135+0000","last_fullsized":"2023-11-24T07:58:39.662135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670177+0000","last_change":"2023-11-24T07:58:38.673409+0000","last_active":"2023-11-24T07:58:39.670177+0000","last_peered":"2023-11-24T07:58:39.670177+0000","last_clean":"2023-11-24T07:58:39.670177+0000","last_became_active":"2023-11-24T07:58:38.673109+0000","last_became_peered":"2023-11-24T07:58:38.673109+0000","last_unstale":"2023-11-24T07:58:39.670177+0000","last_undegraded":"2023-11-24T07:58:39.670177+0000","last_fullsized":"2023-11-24T07:58:39.670177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661303+0000","last_change":"2023-11-24T07:58:38.666773+0000","last_active":"2023-11-24T07:58:39.661303+0000","last_peered":"2023-11-24T07:58:39.661303+0000","last_clean":"2023-11-24T07:58:39.661303+0000","last_became_active":"2023-11-24T07:58:38.666430+0000","last_became_peered":"2023-11-24T07:58:38.666430+0000","last_unstale":"2023-11-24T07:58:39.661303+0000","last_undegraded":"2023-11-24T07:58:39.661303+0000","last_fullsized":"2023-11-24T07:58:39.661303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:43.203 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:58:43.204 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T07:58:43.204 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T07:58:43.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T07:58:43.204 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T07:58:43.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.362+0000 7f3459344700 1 Processor -- start 2023-11-24T07:58:43.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.363+0000 7f3459344700 1 -- start start 2023-11-24T07:58:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.363+0000 7f3459344700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f345406c710 0x7f34540b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.363+0000 7f3459344700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34540ae430 con 0x7f345415e080 2023-11-24T07:58:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.363+0000 7f3459344700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34540ae710 con 0x7f34540af4d0 2023-11-24T07:58:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.363+0000 7f3459344700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34540af0b0 con 0x7f345406c710 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.363+0000 7f34527fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f345406c710 0x7f34540b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f34527fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f345406c710 0x7f34540b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56448/0 (socket says 172.21.15.32:56448) 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f34527fc700 1 -- 172.21.15.32:0/4219281505 learned_addr learned my addr 172.21.15.32:0/4219281505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f34527fc700 1 -- 172.21.15.32:0/4219281505 >> v1:172.21.15.123:6789/0 conn(0x7f34540af4d0 legacy=0x7f345406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f34527fc700 1 -- 172.21.15.32:0/4219281505 >> v1:172.21.15.32:6789/0 conn(0x7f345415e080 legacy=0x7f345406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f3451ffb700 1 -- 172.21.15.32:0/4219281505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837221629 0 0) 0x7f34540ae430 con 0x7f345415e080 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f34527fc700 1 -- 172.21.15.32:0/4219281505 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540ae9f0 con 0x7f345406c710 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f34527fc700 1 --2- 172.21.15.32:0/4219281505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345406c710 0x7f34540b5050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34440098f0 tx=0x7f3444009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6827504439a8d53 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f3451ffb700 1 -- 172.21.15.32:0/4219281505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344400b8a0 con 0x7f345406c710 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f3451ffb700 1 -- 172.21.15.32:0/4219281505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344400ba00 con 0x7f345406c710 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.364+0000 7f3451ffb700 1 -- 172.21.15.32:0/4219281505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344400bd90 con 0x7f345406c710 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 -- 172.21.15.32:0/4219281505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345406c710 msgr2=0x7f34540b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 --2- 172.21.15.32:0/4219281505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345406c710 0x7f34540b5050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34440098f0 tx=0x7f3444009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 -- 172.21.15.32:0/4219281505 shutdown_connections 2023-11-24T07:58:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 --2- 172.21.15.32:0/4219281505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f345406c710 0x7f34540b5050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 -- 172.21.15.32:0/4219281505 >> 172.21.15.32:0/4219281505 conn(0x7f3454071b90 msgr2=0x7f3454071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 -- 172.21.15.32:0/4219281505 shutdown_connections 2023-11-24T07:58:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.365+0000 7f3459344700 1 -- 172.21.15.32:0/4219281505 wait complete. 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.366+0000 7f3459344700 1 Processor -- start 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.366+0000 7f3459344700 1 -- start start 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3459344700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345406c710 0x7f3454157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3459344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 0x7f3454151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3459344700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f345415e080 0x7f34540ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3459344700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34540d0730 con 0x7f34540af4d0 2023-11-24T07:58:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3459344700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3454158490 con 0x7f345406c710 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3459344700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345415f710 con 0x7f345415e080 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 0x7f3454151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 0x7f3454151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56454/0 (socket says 172.21.15.32:56454) 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 -- 172.21.15.32:0/2135991201 learned_addr learned my addr 172.21.15.32:0/2135991201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f3452ffd700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345406c710 0x7f3454157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34537fe700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f345415e080 0x7f34540ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 -- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f345415e080 msgr2=0x7f34540ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f345415e080 0x7f34540ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 -- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345406c710 msgr2=0x7f3454157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345406c710 0x7f3454157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 -- 172.21.15.32:0/2135991201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444009550 con 0x7f34540af4d0 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34537fe700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f345415e080 0x7f34540ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.367+0000 7f34527fc700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 0x7f3454151140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f345409f040 tx=0x7f3444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.368+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444011970 con 0x7f34540af4d0 2023-11-24T07:58:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.368+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344401f070 con 0x7f34540af4d0 2023-11-24T07:58:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.368+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444011cd0 con 0x7f34540af4d0 2023-11-24T07:58:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.368+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345415f920 con 0x7f34540af4d0 2023-11-24T07:58:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.368+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34541601c0 con 0x7f34540af4d0 2023-11-24T07:58:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.369+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345415fb30 con 0x7f34540af4d0 2023-11-24T07:58:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.369+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3444026480 con 0x7f34540af4d0 2023-11-24T07:58:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.369+0000 7f343b7fe700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f344006aed0 0x7f344006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.369+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3444021070 con 0x7f34540af4d0 2023-11-24T07:58:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.370+0000 7f3452ffd700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f344006aed0 0x7f344006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.370+0000 7f3452ffd700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f344006aed0 0x7f344006d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3448007fd0 tx=0x7f3448009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.374+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34440560b0 con 0x7f34540af4d0 2023-11-24T07:58:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.634+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f34540753f0 con 0x7f34540af4d0 2023-11-24T07:58:43.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.637+0000 7f343b7fe700 1 -- 172.21.15.32:0/2135991201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7f3444015090 con 0x7f34540af4d0 2023-11-24T07:58:43.641 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f344006aed0 msgr2=0x7f344006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f344006aed0 0x7f344006d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3448007fd0 tx=0x7f3448009040 comp rx=0 tx=0).stop 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 msgr2=0x7f3454151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 0x7f3454151140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f345409f040 tx=0x7f3444004750 comp rx=0 tx=0).stop 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 shutdown_connections 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f344006aed0 0x7f344006d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f345415e080 0x7f34540ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.639+0000 7f3459344700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345406c710 0x7f3454157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.640+0000 7f3459344700 1 --2- 172.21.15.32:0/2135991201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34540af4d0 0x7f3454151140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.640+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 >> 172.21.15.32:0/2135991201 conn(0x7f3454071b90 msgr2=0x7f34540b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.640+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 shutdown_connections 2023-11-24T07:58:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.640+0000 7f3459344700 1 -- 172.21.15.32:0/2135991201 wait complete. 2023-11-24T07:58:43.653 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1302929475},{"type":"v1","addr":"172.21.15.123:6833","nonce":1302929475}]},"active_addr":"172.21.15.123:6833/1302929475","active_change":"2023-11-24T07:58:26.521793+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2156780906}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3956559345}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1328145929}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3223130902}]}]} 2023-11-24T07:58:43.659 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T07:58:43.659 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T07:58:43.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:43.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.781+0000 7f39b8341700 1 Processor -- start 2023-11-24T07:58:43.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.782+0000 7f39b8341700 1 -- start start 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.782+0000 7f39b8341700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39b015e080 0x7f39b00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.782+0000 7f39b8341700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f39b00af4d0 0x7f39b00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b8341700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b00ae430 con 0x7f39b006c000 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b8341700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b00af0b0 con 0x7f39b015e080 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b8341700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b00b05a0 con 0x7f39b00af4d0 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39b015e080 0x7f39b00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39b015e080 0x7f39b00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56460/0 (socket says 172.21.15.32:56460) 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 -- 172.21.15.32:0/486173531 learned_addr learned my addr 172.21.15.32:0/486173531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b68de700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39b006c000 0x7f39b0073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58176/0 (socket says 172.21.15.32:58176) 2023-11-24T07:58:43.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b58dc700 1 --2- 172.21.15.32:0/486173531 >> v2:172.21.15.123:3300/0 conn(0x7f39b00af4d0 0x7f39b00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 -- 172.21.15.32:0/486173531 >> v2:172.21.15.123:3300/0 conn(0x7f39b00af4d0 msgr2=0x7f39b00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 --2- 172.21.15.32:0/486173531 >> v2:172.21.15.123:3300/0 conn(0x7f39b00af4d0 0x7f39b00afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 -- 172.21.15.32:0/486173531 >> v1:172.21.15.32:6789/0 conn(0x7f39b006c000 legacy=0x7f39b0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 -- 172.21.15.32:0/486173531 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b00ae710 con 0x7f39b015e080 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b50db700 1 -- 172.21.15.32:0/486173531 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558475849 0 0) 0x7f39b00ae430 con 0x7f39b006c000 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b58dc700 1 --2- 172.21.15.32:0/486173531 >> v2:172.21.15.123:3300/0 conn(0x7f39b00af4d0 0x7f39b00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b60dd700 1 --2- 172.21.15.32:0/486173531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b015e080 0x7f39b00b5050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39b006e4e0 tx=0x7f39ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d180da8cff18bb25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b50db700 1 -- 172.21.15.32:0/486173531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00b860 con 0x7f39b015e080 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b50db700 1 -- 172.21.15.32:0/486173531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00b9c0 con 0x7f39b015e080 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.783+0000 7f39b50db700 1 -- 172.21.15.32:0/486173531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00bd50 con 0x7f39b015e080 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 -- 172.21.15.32:0/486173531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b015e080 msgr2=0x7f39b00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 --2- 172.21.15.32:0/486173531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b015e080 0x7f39b00b5050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39b006e4e0 tx=0x7f39ac009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 -- 172.21.15.32:0/486173531 shutdown_connections 2023-11-24T07:58:43.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 --2- 172.21.15.32:0/486173531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b015e080 0x7f39b00b5050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 --2- 172.21.15.32:0/486173531 >> v2:172.21.15.123:3300/0 conn(0x7f39b00af4d0 0x7f39b00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 -- 172.21.15.32:0/486173531 >> 172.21.15.32:0/486173531 conn(0x7f39b0071b90 msgr2=0x7f39b0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:43.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 -- 172.21.15.32:0/486173531 shutdown_connections 2023-11-24T07:58:43.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.784+0000 7f39b8341700 1 -- 172.21.15.32:0/486173531 wait complete. 2023-11-24T07:58:43.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.785+0000 7f39b8341700 1 Processor -- start 2023-11-24T07:58:43.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.786+0000 7f39b8341700 1 -- start start 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.786+0000 7f39b8341700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 0x7f39b013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.786+0000 7f39b8341700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39b00af4d0 0x7f39b015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b8341700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39b015e080 0x7f39b0153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b8341700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b015ef70 con 0x7f39b006c000 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b8341700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b015f250 con 0x7f39b015e080 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b8341700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b015f530 con 0x7f39b00af4d0 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 0x7f39b013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 0x7f39b013e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56466/0 (socket says 172.21.15.32:56466) 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 -- 172.21.15.32:0/2624855157 learned_addr learned my addr 172.21.15.32:0/2624855157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b68de700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39b015e080 0x7f39b0153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b58dc700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39b00af4d0 0x7f39b015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 -- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39b00af4d0 msgr2=0x7f39b015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39b00af4d0 0x7f39b015e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 -- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39b015e080 msgr2=0x7f39b0153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39b015e080 0x7f39b0153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 -- 172.21.15.32:0/2624855157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac009550 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b60dd700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 0x7f39b013e2c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009870 tx=0x7f39ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0038a0 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac01e070 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b015f810 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac003c40 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.787+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b014abe0 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.788+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b00753f0 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.789+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39ac025480 con 0x7f39b006c000 2023-11-24T07:58:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.789+0000 7f39a2ffd700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f399406aed0 0x7f399406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.789+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39ac020070 con 0x7f39b006c000 2023-11-24T07:58:43.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.789+0000 7f39b58dc700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f399406aed0 0x7f399406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.790+0000 7f39b58dc700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f399406aed0 0x7f399406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39b0156d60 tx=0x7f39a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.794+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39ac055c90 con 0x7f39b006c000 2023-11-24T07:58:43.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.997+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39b00d09b0 con 0x7f39b006c000 2023-11-24T07:58:44.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:43.998+0000 7f39a2ffd700 1 -- 172.21.15.32:0/2624855157 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f39ac015070 con 0x7f39b006c000 2023-11-24T07:58:44.000 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:44.000 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.000+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f399406aed0 msgr2=0x7f399406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.000+0000 7f39b8341700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f399406aed0 0x7f399406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39b0156d60 tx=0x7f39a4008040 comp rx=0 tx=0).stop 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.000+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 msgr2=0x7f39b013e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.000+0000 7f39b8341700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 0x7f39b013e2c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009870 tx=0x7f39ac004750 comp rx=0 tx=0).stop 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 shutdown_connections 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f399406aed0 0x7f399406d390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39b015e080 0x7f39b0153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39b006c000 0x7f39b013e2c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 --2- 172.21.15.32:0/2624855157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39b00af4d0 0x7f39b015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 >> 172.21.15.32:0/2624855157 conn(0x7f39b0071b90 msgr2=0x7f39b00b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 shutdown_connections 2023-11-24T07:58:44.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.001+0000 7f39b8341700 1 -- 172.21.15.32:0/2624855157 wait complete. 2023-11-24T07:58:44.015 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T07:58:44.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.137+0000 7f3e185a8700 1 Processor -- start 2023-11-24T07:58:44.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.137+0000 7f3e185a8700 1 -- start start 2023-11-24T07:58:44.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e185a8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3e101446a0 0x7f3e100b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e185a8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e100af950 con 0x7f3e100adf00 2023-11-24T07:58:44.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e185a8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e100afc30 con 0x7f3e1006c710 2023-11-24T07:58:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e185a8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e10144a80 con 0x7f3e101446a0 2023-11-24T07:58:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e15b43700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3e100adf00 0x7f3e1006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58188/0 (socket says 172.21.15.32:58188) 2023-11-24T07:58:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e15b43700 1 -- 172.21.15.32:0/1351033594 learned_addr learned my addr 172.21.15.32:0/1351033594 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e16344700 1 --2- 172.21.15.32:0/1351033594 >> v2:172.21.15.123:3300/0 conn(0x7f3e101446a0 0x7f3e100b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.138+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987587695 0 0) 0x7f3e100af950 con 0x7f3e100adf00 2023-11-24T07:58:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc0035b0 con 0x7f3e100adf00 2023-11-24T07:58:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3433289526 0 0) 0x7f3dfc0035b0 con 0x7f3e100adf00 2023-11-24T07:58:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 >> v2:172.21.15.123:3300/0 conn(0x7f3e101446a0 msgr2=0x7f3e100b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 --2- 172.21.15.32:0/1351033594 >> v2:172.21.15.123:3300/0 conn(0x7f3e101446a0 0x7f3e100b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 >> v1:172.21.15.123:6789/0 conn(0x7f3e1006c710 legacy=0x7f3e100b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100aff10 con 0x7f3e100adf00 2023-11-24T07:58:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f3e0c001670 con 0x7f3e100adf00 2023-11-24T07:58:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 >> v1:172.21.15.32:6789/0 conn(0x7f3e100adf00 legacy=0x7f3e1006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dfc003a00 0x7f3dfc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3dfc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dfc005210 0x7f3dfc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10144a80 con 0x7f3dfc004850 2023-11-24T07:58:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc00c1b0 con 0x7f3dfc005210 2023-11-24T07:58:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc00c370 con 0x7f3dfc003a00 2023-11-24T07:58:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3dfc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e16344700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dfc003a00 0x7f3dfc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e16b45700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dfc005210 0x7f3dfc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 -- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dfc003a00 msgr2=0x7f3dfc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dfc003a00 0x7f3dfc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 -- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dfc005210 msgr2=0x7f3dfc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dfc005210 0x7f3dfc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e16b45700 1 -- 172.21.15.32:0/1351033594 reap_dead start 2023-11-24T07:58:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 -- 172.21.15.32:0/1351033594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0035b0 con 0x7f3dfc004850 2023-11-24T07:58:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.139+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3dfc004cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e1006f1d0 tx=0x7f3dfc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32076bd1425215ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00cdd0 con 0x7f3dfc004850 2023-11-24T07:58:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c005960 con 0x7f3dfc004850 2023-11-24T07:58:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e15342700 1 -- 172.21.15.32:0/1351033594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c005cd0 con 0x7f3dfc004850 2023-11-24T07:58:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 -- 172.21.15.32:0/1351033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 msgr2=0x7f3dfc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3dfc004cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e1006f1d0 tx=0x7f3dfc005880 comp rx=0 tx=0).stop 2023-11-24T07:58:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 -- 172.21.15.32:0/1351033594 shutdown_connections 2023-11-24T07:58:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1351033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3dfc004cd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 -- 172.21.15.32:0/1351033594 >> 172.21.15.32:0/1351033594 conn(0x7f3e10071b90 msgr2=0x7f3e10071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 -- 172.21.15.32:0/1351033594 shutdown_connections 2023-11-24T07:58:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.140+0000 7f3e185a8700 1 -- 172.21.15.32:0/1351033594 wait complete. 2023-11-24T07:58:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.141+0000 7f3e185a8700 1 Processor -- start 2023-11-24T07:58:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.141+0000 7f3e185a8700 1 -- start start 2023-11-24T07:58:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e185a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3e101f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e185a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e10143510 0x7f3e101f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e185a8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e101f1590 0x7f3e10141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e185a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101f4b10 con 0x7f3dfc004850 2023-11-24T07:58:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e185a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101f5970 con 0x7f3e10143510 2023-11-24T07:58:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e185a8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101f48e0 con 0x7f3e101f1590 2023-11-24T07:58:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.142+0000 7f3e16344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3e101f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3e101f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56486/0 (socket says 172.21.15.32:56486) 2023-11-24T07:58:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 -- 172.21.15.32:0/1244408934 learned_addr learned my addr 172.21.15.32:0/1244408934 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16b45700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e101f1590 0x7f3e10141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e10143510 0x7f3e101f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 -- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e101f1590 msgr2=0x7f3e10141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e101f1590 0x7f3e10141150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 -- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e10143510 msgr2=0x7f3e101f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e10143510 0x7f3e101f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 -- 172.21.15.32:0/1244408934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e101f4d60 con 0x7f3dfc004850 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e10143510 0x7f3e101f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e16344700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3e101f3370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3e04011aa0 tx=0x7f3e04011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400f9e0 con 0x7f3dfc004850 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0400fb40 con 0x7f3dfc004850 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e101f5040 con 0x7f3dfc004850 2023-11-24T07:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04014440 con 0x7f3dfc004850 2023-11-24T07:58:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.143+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e101f5600 con 0x7f3dfc004850 2023-11-24T07:58:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.144+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e100753f0 con 0x7f3dfc004850 2023-11-24T07:58:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.145+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e040145a0 con 0x7f3dfc004850 2023-11-24T07:58:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.145+0000 7f3e02ffd700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3df406aed0 0x7f3df406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.145+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e04089d60 con 0x7f3dfc004850 2023-11-24T07:58:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.145+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3df406aed0 0x7f3df406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.146+0000 7f3e15b43700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3df406aed0 0x7f3df406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e1006f1d0 tx=0x7f3e0c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.150+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e04054ea0 con 0x7f3dfc004850 2023-11-24T07:58:44.163 INFO:tasks.ceph.mds.d.smithi123.stderr:2023-11-24T07:58:44.153+0000 7f449abe0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:58:44.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.352+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e10067f60 con 0x7f3dfc004850 2023-11-24T07:58:44.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.353+0000 7f3e02ffd700 1 -- 172.21.15.32:0/1244408934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f3e040584c0 con 0x7f3dfc004850 2023-11-24T07:58:44.355 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:44.355 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3df406aed0 msgr2=0x7f3df406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3df406aed0 0x7f3df406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e1006f1d0 tx=0x7f3e0c007000 comp rx=0 tx=0).stop 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 msgr2=0x7f3e101f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3e101f3370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3e04011aa0 tx=0x7f3e04011e60 comp rx=0 tx=0).stop 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 shutdown_connections 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3df406aed0 0x7f3df406d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e101f1590 0x7f3e10141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc004850 0x7f3e101f3370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 --2- 172.21.15.32:0/1244408934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e10143510 0x7f3e101f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 >> 172.21.15.32:0/1244408934 conn(0x7f3e10071b90 msgr2=0x7f3e1015e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 shutdown_connections 2023-11-24T07:58:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.356+0000 7f3e185a8700 1 -- 172.21.15.32:0/1244408934 wait complete. 2023-11-24T07:58:44.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T07:58:44.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T07:58:44.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T07:58:44.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T07:58:44.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T07:58:44.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T07:58:44.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T07:58:44.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T07:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.592+0000 7f7ab96ee700 1 Processor -- start 2023-11-24T07:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.592+0000 7f7ab96ee700 1 -- start start 2023-11-24T07:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.593+0000 7f7ab96ee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ab40b4260 0x7f7ab415d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.593+0000 7f7ab96ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab415d970 con 0x7f7ab40d0520 2023-11-24T07:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.593+0000 7f7ab96ee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab40723c0 con 0x7f7ab40714d0 2023-11-24T07:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.593+0000 7f7ab96ee700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab4072d60 con 0x7f7ab40b4260 2023-11-24T07:58:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.594+0000 7f3dafb75700 1 Processor -- start 2023-11-24T07:58:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.594+0000 7f25f81f9700 1 Processor -- start 2023-11-24T07:58:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f7ab37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7ab40714d0 0x7f7ab4140fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:44964/0 (socket says 172.21.15.32:44964) 2023-11-24T07:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f7ab37fe700 1 -- 172.21.15.32:0/3200193690 learned_addr learned my addr 172.21.15.32:0/3200193690 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.594+0000 7f3dafb75700 1 -- start start 2023-11-24T07:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f3dafb75700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3da80cde90 0x7f3da80ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f3dafb75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80b07e0 0x7f3da80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f3dafb75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da8158620 con 0x7f3da815abe0 2023-11-24T07:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f3dafb75700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da815bc40 con 0x7f3da80cde90 2023-11-24T07:58:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.595+0000 7f3dafb75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da815b960 con 0x7f3da80b07e0 2023-11-24T07:58:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f3dad911700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3da80cde90 0x7f3da80ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f3dad911700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3da80cde90 0x7f3da80ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40650/0 (socket says 172.21.15.32:40650) 2023-11-24T07:58:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f3dad911700 1 -- 172.21.15.32:0/408884082 learned_addr learned my addr 172.21.15.32:0/408884082 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283866180 0 0) 0x7f7ab40723c0 con 0x7f7ab40714d0 2023-11-24T07:58:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f3dae112700 1 --2- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80b07e0 0x7f3da80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a9c0035b0 con 0x7f7ab40714d0 2023-11-24T07:58:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951006512 0 0) 0x7f7ab415d970 con 0x7f7ab40d0520 2023-11-24T07:58:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab40723c0 con 0x7f7ab40d0520 2023-11-24T07:58:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab27fc700 1 --2- 172.21.15.32:0/3200193690 >> v2:172.21.15.123:3300/0 conn(0x7f7ab40b4260 0x7f7ab415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421185303 0 0) 0x7f7a9c0035b0 con 0x7f7ab40714d0 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 >> v2:172.21.15.123:3300/0 conn(0x7f7ab40b4260 msgr2=0x7f7ab415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 --2- 172.21.15.32:0/3200193690 >> v2:172.21.15.123:3300/0 conn(0x7f7ab40b4260 0x7f7ab415d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 >> v1:172.21.15.32:6789/0 conn(0x7f7ab40d0520 legacy=0x7f7ab40b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4160ba0 con 0x7f7ab40714d0 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f7aac0019f0 con 0x7f7ab40714d0 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.594+0000 7f1dca709700 1 Processor -- start 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.594+0000 7f1dca709700 1 -- start start 2023-11-24T07:58:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dca709700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1dc40b30d0 0x7f1dc40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dca709700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc40b3640 con 0x7f1dc40b5510 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dca709700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc40b3920 con 0x7f1dc4159f60 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dca709700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc40b59c0 con 0x7f1dc40b30d0 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dc3fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1dc4159f60 0x7f1dc415a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:44972/0 (socket says 172.21.15.32:44972) 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dc3fff700 1 -- 172.21.15.32:0/1263039923 learned_addr learned my addr 172.21.15.32:0/1263039923 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dc37fe700 1 --2- 172.21.15.32:0/1263039923 >> v2:172.21.15.123:3300/0 conn(0x7f1dc40b30d0 0x7f1dc40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 >> v1:172.21.15.123:6789/0 conn(0x7f7ab40714d0 legacy=0x7f7ab4140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c004c40 0x7f7a9c005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a9c0037f0 0x7f7a9c003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7a9c00ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab4072d60 con 0x7f7a9c004c40 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00c260 con 0x7f7a9c0042d0 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00c420 con 0x7f7a9c0037f0 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dad911700 1 -- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80b07e0 msgr2=0x7f3da80b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dad911700 1 --2- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80b07e0 0x7f3da80b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dad911700 1 -- 172.21.15.32:0/408884082 >> v1:172.21.15.32:6789/0 conn(0x7f3da815abe0 legacy=0x7f3da815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dad911700 1 -- 172.21.15.32:0/408884082 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80afd00 con 0x7f3da80cde90 2023-11-24T07:58:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab27fc700 1 --2- 172.21.15.32:0/3200193690 >> v2:172.21.15.123:3300/0 conn(0x7f7ab40b4260 0x7f7ab415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7a9c00ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab27fc700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a9c0037f0 0x7f7a9c003c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab2ffd700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c004c40 0x7f7a9c005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dad911700 1 --2- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80cde90 0x7f3da80ce270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0098f0 tx=0x7f3d9c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cac312ac6f1519f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 -- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a9c0037f0 msgr2=0x7f7a9c003c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a9c0037f0 0x7f7a9c003c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 -- 172.21.15.32:0/3200193690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c004c40 msgr2=0x7f7a9c005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c004c40 0x7f7a9c005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 -- 172.21.15.32:0/3200193690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4160ba0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab2ffd700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c004c40 0x7f7a9c005040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 -- 172.21.15.32:0/3200193690 reap_dead start 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dac90f700 1 -- 172.21.15.32:0/408884082 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00b8a0 con 0x7f3da80cde90 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dac90f700 1 -- 172.21.15.32:0/408884082 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00ba00 con 0x7f3da80cde90 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f3dac90f700 1 -- 172.21.15.32:0/408884082 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00bd90 con 0x7f3da80cde90 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809083605 0 0) 0x7f1dc40b3640 con 0x7f1dc40b5510 2023-11-24T07:58:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dac0035b0 con 0x7f1dc40b5510 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916035700 0 0) 0x7f1dac0035b0 con 0x7f1dc40b5510 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 >> v2:172.21.15.123:3300/0 conn(0x7f1dc40b30d0 msgr2=0x7f1dc40b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 --2- 172.21.15.32:0/1263039923 >> v2:172.21.15.123:3300/0 conn(0x7f1dc40b30d0 0x7f1dc40b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 >> v1:172.21.15.123:6789/0 conn(0x7f1dc4159f60 legacy=0x7f1dc415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40b3c00 con 0x7f1dc40b5510 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f1db4001670 con 0x7f1dc40b5510 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 >> v1:172.21.15.32:6789/0 conn(0x7f1dc40b5510 legacy=0x7f1dc4071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dac003a50 0x7f1dac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac004850 0x7f1dac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac005210 0x7f1dac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc40b59c0 con 0x7f1dac003a50 2023-11-24T07:58:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac00c110 con 0x7f1dac004850 2023-11-24T07:58:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac00c2d0 con 0x7f1dac005210 2023-11-24T07:58:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f7ab37fe700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7a9c00ba90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00a860 tx=0x7f7a9c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce43e9926030429 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f1dc8ca6700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac005210 0x7f1dac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00a310 con 0x7f7a9c0042d0 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac012620 con 0x7f7a9c0042d0 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f7ab1ffb700 1 -- 172.21.15.32:0/3200193690 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0129d0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f25f81f9700 1 -- start start 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f81f9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25f0166660 0x7f25f00b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f81f9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f00af6d0 0x7f25f0071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f81f9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f0071910 con 0x7f25f0162220 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f81f9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f00afb80 con 0x7f25f0166660 2023-11-24T07:58:44.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f81f9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0070fe0 con 0x7f25f00af6d0 2023-11-24T07:58:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dac003a50 0x7f1dac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 -- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac005210 msgr2=0x7f1dac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac005210 0x7f1dac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 -- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac004850 msgr2=0x7f1dac004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:58:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac004850 0x7f1dac004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 -- 172.21.15.32:0/1263039923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40b3c00 con 0x7f1dac003a50 2023-11-24T07:58:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.596+0000 7f30807d9700 1 Processor -- start 2023-11-24T07:58:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.597+0000 7f30807d9700 1 -- start start 2023-11-24T07:58:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f30807d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 0x7f30780b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f30807d9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f307806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f5794700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25f0166660 0x7f25f00b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f5794700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25f0166660 0x7f25f00b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40674/0 (socket says 172.21.15.32:40674) 2023-11-24T07:58:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f25f5794700 1 -- 172.21.15.32:0/3797597158 learned_addr learned my addr 172.21.15.32:0/3797597158 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f3dafb75700 1 -- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80cde90 msgr2=0x7f3da80ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f3dafb75700 1 --2- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80cde90 0x7f3da80ce270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0098f0 tx=0x7f3d9c009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f3dafb75700 1 -- 172.21.15.32:0/408884082 shutdown_connections 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f3dafb75700 1 --2- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da80b07e0 0x7f3da80b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f3dafb75700 1 --2- 172.21.15.32:0/408884082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80cde90 0x7f3da80ce270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f3dafb75700 1 -- 172.21.15.32:0/408884082 >> 172.21.15.32:0/408884082 conn(0x7f3da8072610 msgr2=0x7f3da8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f3dafb75700 1 -- 172.21.15.32:0/408884082 shutdown_connections 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f7ab96ee700 1 -- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 msgr2=0x7f7a9c00ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f7ab96ee700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7a9c00ba90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00a860 tx=0x7f7a9c005880 comp rx=0 tx=0).stop 2023-11-24T07:58:44.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f3dafb75700 1 -- 172.21.15.32:0/408884082 wait complete. 2023-11-24T07:58:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dac003a50 0x7f1dac003e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc002070 tx=0x7f1dac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1439b0505de67f2b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f25f5794700 1 -- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f00af6d0 msgr2=0x7f25f0071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f25f5794700 1 --2- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f00af6d0 0x7f25f0071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f25f5794700 1 -- 172.21.15.32:0/3797597158 >> v1:172.21.15.123:6789/0 conn(0x7f25f0162220 legacy=0x7f25f0165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f25f5794700 1 -- 172.21.15.32:0/3797597158 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f00712c0 con 0x7f25f0166660 2023-11-24T07:58:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f1dc8ca6700 1 -- 172.21.15.32:0/1263039923 reap_dead start 2023-11-24T07:58:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f30807d9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307806ced0 con 0x7f3078163a80 2023-11-24T07:58:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f30807d9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307806da70 con 0x7f30780b3b50 2023-11-24T07:58:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.598+0000 7f30807d9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30780b4470 con 0x7f307806d4d0 2023-11-24T07:58:44.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f25f5794700 1 --2- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 0x7f25f00b5190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098f0 tx=0x7f25ec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d50358ae497f7ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc01b600 con 0x7f1dac003a50 2023-11-24T07:58:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307e575700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 0x7f30780b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307e575700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 0x7f30780b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56528/0 (socket says 172.21.15.32:56528) 2023-11-24T07:58:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307e575700 1 -- 172.21.15.32:0/3505729045 learned_addr learned my addr 172.21.15.32:0/3505729045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307dd74700 1 --2- 172.21.15.32:0/3505729045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f307806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc01b760 con 0x7f1dac003a50 2023-11-24T07:58:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f1dc2ffd700 1 -- 172.21.15.32:0/1263039923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc01bad0 con 0x7f1dac003a50 2023-11-24T07:58:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307dd74700 1 -- 172.21.15.32:0/3505729045 >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 msgr2=0x7f30780b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307dd74700 1 --2- 172.21.15.32:0/3505729045 >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 0x7f30780b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307dd74700 1 -- 172.21.15.32:0/3505729045 >> v1:172.21.15.123:6789/0 conn(0x7f3078163a80 legacy=0x7f3078163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.599+0000 7f307dd74700 1 -- 172.21.15.32:0/3505729045 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30781634e0 con 0x7f307806d4d0 2023-11-24T07:58:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f307e575700 1 --2- 172.21.15.32:0/3505729045 >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 0x7f30780b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f307dd74700 1 --2- 172.21.15.32:0/3505729045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f307806eee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3068009840 tx=0x7f3068009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b3f7eafde6257 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f1dca709700 1 -- 172.21.15.32:0/1263039923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dac003a50 msgr2=0x7f1dac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f1dca709700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dac003a50 0x7f1dac003e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc002070 tx=0x7f1dac005880 comp rx=0 tx=0).stop 2023-11-24T07:58:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f307d573700 1 -- 172.21.15.32:0/3505729045 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068011830 con 0x7f307806d4d0 2023-11-24T07:58:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f307d573700 1 -- 172.21.15.32:0/3505729045 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3068011990 con 0x7f307806d4d0 2023-11-24T07:58:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f307d573700 1 -- 172.21.15.32:0/3505729045 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068011d20 con 0x7f307806d4d0 2023-11-24T07:58:44.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f3dafb75700 1 Processor -- start 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f7ab96ee700 1 -- 172.21.15.32:0/3200193690 shutdown_connections 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f7ab96ee700 1 --2- 172.21.15.32:0/3200193690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7a9c00ba90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f7ab96ee700 1 -- 172.21.15.32:0/3200193690 >> 172.21.15.32:0/3200193690 conn(0x7f7ab40cfa30 msgr2=0x7f7ab40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f7ab96ee700 1 -- 172.21.15.32:0/3200193690 shutdown_connections 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f25f4f93700 1 -- 172.21.15.32:0/3797597158 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00b8a0 con 0x7f25f0166660 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f25f4f93700 1 -- 172.21.15.32:0/3797597158 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec00ba00 con 0x7f25f0166660 2023-11-24T07:58:44.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f25f4f93700 1 -- 172.21.15.32:0/3797597158 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00bd90 con 0x7f25f0166660 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f30807d9700 1 -- 172.21.15.32:0/3505729045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 msgr2=0x7f307806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f30807d9700 1 --2- 172.21.15.32:0/3505729045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f307806eee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3068009840 tx=0x7f3068009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f1dca709700 1 -- 172.21.15.32:0/1263039923 shutdown_connections 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f1dca709700 1 --2- 172.21.15.32:0/1263039923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dac003a50 0x7f1dac003e70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f1dca709700 1 -- 172.21.15.32:0/1263039923 >> 172.21.15.32:0/1263039923 conn(0x7f1dc406cec0 msgr2=0x7f1dc406d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f1dca709700 1 -- 172.21.15.32:0/1263039923 shutdown_connections 2023-11-24T07:58:44.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f1dca709700 1 -- 172.21.15.32:0/1263039923 wait complete. 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.600+0000 7f3dafb75700 1 -- start start 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dafb75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80b07e0 0x7f3da81f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dafb75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80cde90 0x7f3da80c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dafb75700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da815abe0 0x7f3da80cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dafb75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80c5850 con 0x7f3da80cde90 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dafb75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da81f8930 con 0x7f3da80b07e0 2023-11-24T07:58:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dafb75700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80b1920 con 0x7f3da815abe0 2023-11-24T07:58:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f7ab96ee700 1 -- 172.21.15.32:0/3200193690 wait complete. 2023-11-24T07:58:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f30807d9700 1 -- 172.21.15.32:0/3505729045 shutdown_connections 2023-11-24T07:58:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f30807d9700 1 --2- 172.21.15.32:0/3505729045 >> v2:172.21.15.32:3300/0 conn(0x7f30780b3b50 0x7f30780b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f30807d9700 1 --2- 172.21.15.32:0/3505729045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f307806eee0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f30807d9700 1 -- 172.21.15.32:0/3505729045 >> 172.21.15.32:0/3505729045 conn(0x7f30780cfa30 msgr2=0x7f30780cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dae112700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da815abe0 0x7f3da80cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dae112700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da815abe0 0x7f3da80cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51788/0 (socket says 172.21.15.32:51788) 2023-11-24T07:58:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.601+0000 7f3dae112700 1 -- 172.21.15.32:0/3579549995 learned_addr learned my addr 172.21.15.32:0/3579549995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dad110700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80cde90 0x7f3da80c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f30807d9700 1 -- 172.21.15.32:0/3505729045 shutdown_connections 2023-11-24T07:58:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f30807d9700 1 -- 172.21.15.32:0/3505729045 wait complete. 2023-11-24T07:58:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dae112700 1 -- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80b07e0 msgr2=0x7f3da81f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dae112700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da80b07e0 0x7f3da81f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dae112700 1 -- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80cde90 msgr2=0x7f3da80c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dae112700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80cde90 0x7f3da80c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dae112700 1 -- 172.21.15.32:0/3579549995 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c009550 con 0x7f3da815abe0 2023-11-24T07:58:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f3dad110700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3da80cde90 0x7f3da80c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.602+0000 7f1dca709700 1 Processor -- start 2023-11-24T07:58:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.603+0000 7f1dca709700 1 -- start start 2023-11-24T07:58:44.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.603+0000 7f7ab96ee700 1 Processor -- start 2023-11-24T07:58:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.603+0000 7f7ab96ee700 1 -- start start 2023-11-24T07:58:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f30807d9700 1 Processor -- start 2023-11-24T07:58:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab96ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7ab413d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab96ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab40cd7e0 0x7f7ab413c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab96ee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab4158ad0 0x7f7ab4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab96ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab41594a0 con 0x7f7ab40cd7e0 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab96ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40cdd50 con 0x7f7a9c0042d0 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab96ee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40ce030 con 0x7f7ab4158ad0 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dca709700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac003a50 0x7f1dc41f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dca709700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc41467b0 0x7f1dc4150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dca709700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dc4146c30 0x7f1dc40b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dca709700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc41f35c0 con 0x7f1dc41467b0 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dca709700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4151810 con 0x7f1dc4146c30 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dca709700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4151af0 con 0x7f1dac003a50 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7ab413d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7ab413d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40694/0 (socket says 172.21.15.32:40694) 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab2ffd700 1 -- 172.21.15.32:0/373919222 learned_addr learned my addr 172.21.15.32:0/373919222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f30807d9700 1 -- start start 2023-11-24T07:58:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f25f81f9700 1 -- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 msgr2=0x7f25f00b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f25f81f9700 1 --2- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 0x7f25f00b5190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098f0 tx=0x7f25ec009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dc8ca6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dc4146c30 0x7f1dc40b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dc8ca6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dc4146c30 0x7f1dc40b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40696/0 (socket says 172.21.15.32:40696) 2023-11-24T07:58:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dc8ca6700 1 -- 172.21.15.32:0/875210696 learned_addr learned my addr 172.21.15.32:0/875210696 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac003a50 0x7f1dc41f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f7ab27fc700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab40cd7e0 0x7f7ab413c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.604+0000 7f1dc37fe700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc41467b0 0x7f1dc4150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f7ab2ffd700 1 -- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab4158ad0 msgr2=0x7f7ab4158f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f7ab2ffd700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ab4158ad0 0x7f7ab4158f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f7ab2ffd700 1 -- 172.21.15.32:0/373919222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab40cd7e0 msgr2=0x7f7ab413c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f7ab2ffd700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ab40cd7e0 0x7f7ab413c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f7ab2ffd700 1 -- 172.21.15.32:0/373919222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40ce310 con 0x7f7a9c0042d0 2023-11-24T07:58:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc3fff700 1 -- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dc4146c30 msgr2=0x7f1dc40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dc4146c30 0x7f1dc40b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc3fff700 1 -- 172.21.15.32:0/875210696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc41467b0 msgr2=0x7f1dc4150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1dc41467b0 0x7f1dc4150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc3fff700 1 -- 172.21.15.32:0/875210696 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc41f2f70 con 0x7f1dac003a50 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc3fff700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac003a50 0x7f1dc41f3a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc019000 tx=0x7f1dbc0013d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc021070 con 0x7f1dac003a50 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc001740 con 0x7f1dac003a50 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc01a820 con 0x7f1dac003a50 2023-11-24T07:58:44.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc41f3250 con 0x7f1dac003a50 2023-11-24T07:58:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f30807d9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f30781473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f30807d9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30780b3b50 0x7f30781593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f30807d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078163a80 0x7f307815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f30807d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307815a8c0 con 0x7f3078163a80 2023-11-24T07:58:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f30807d9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3078159d40 con 0x7f30780b3b50 2023-11-24T07:58:44.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f30807d9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307815a020 con 0x7f307806d4d0 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30780b3b50 0x7f30781593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307e575700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f30781473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30780b3b50 0x7f30781593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40704/0 (socket says 172.21.15.32:40704) 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 -- 172.21.15.32:0/1924094833 learned_addr learned my addr 172.21.15.32:0/1924094833 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307ed76700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078163a80 0x7f307815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f7ab2ffd700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7ab413d1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800e580 tx=0x7f7aa800e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f3dae112700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da815abe0 0x7f3da80cbad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3da0007e40 tx=0x7f3da000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 -- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 msgr2=0x7f30781473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f307806d4d0 0x7f30781473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 -- 172.21.15.32:0/1924094833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078163a80 msgr2=0x7f307815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3078163a80 0x7f307815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f307dd74700 1 -- 172.21.15.32:0/1924094833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009550 con 0x7f30780b3b50 2023-11-24T07:58:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.605+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc41512b0 con 0x7f1dac003a50 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f25f81f9700 1 -- 172.21.15.32:0/3797597158 shutdown_connections 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f25f81f9700 1 --2- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f00af6d0 0x7f25f0071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f25f81f9700 1 --2- 172.21.15.32:0/3797597158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 0x7f25f00b5190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f25f81f9700 1 -- 172.21.15.32:0/3797597158 >> 172.21.15.32:0/3797597158 conn(0x7f25f006cec0 msgr2=0x7f25f006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.607+0000 7f25f81f9700 1 -- 172.21.15.32:0/3797597158 shutdown_connections 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000eea0 con 0x7f3da815abe0 2023-11-24T07:58:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da0014420 con 0x7f3da815abe0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00147d0 con 0x7f3da815abe0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da81535d0 con 0x7f3da815abe0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da80c7910 con 0x7f3da815abe0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80052e0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa8005440 con 0x7f7a9c0042d0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80038a0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab40ce5f0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.607+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab415bce0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ab4072980 con 0x7f7a9c0042d0 2023-11-24T07:58:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f307dd74700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30780b3b50 0x7f30781593d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fd0 tx=0x7f3068004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.606+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1dc40b41c0 con 0x7f1dac003a50 2023-11-24T07:58:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1dbc01a980 con 0x7f1dac003a50 2023-11-24T07:58:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7fe5e5b98700 1 Processor -- start 2023-11-24T07:58:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f1dc0ff9700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1db006aed0 0x7f1db006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dbc08fc40 con 0x7f1dac003a50 2023-11-24T07:58:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7f25f81f9700 1 -- 172.21.15.32:0/3797597158 wait complete. 2023-11-24T07:58:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f1dc0ff9700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7f1db0073070 0x7f1db0075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 --> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1db0075b40 con 0x7f1db0073070 2023-11-24T07:58:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dbc090090 con 0x7f1dac003a50 2023-11-24T07:58:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7aa8003a00 con 0x7f7a9c0042d0 2023-11-24T07:58:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f1dc37fe700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1db006aed0 0x7f1db006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30680036a0 con 0x7f30780b3b50 2023-11-24T07:58:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306801e070 con 0x7f30780b3b50 2023-11-24T07:58:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068003a60 con 0x7f30780b3b50 2023-11-24T07:58:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7efd42edd700 1 Processor -- start 2023-11-24T07:58:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f25f81f9700 1 Processor -- start 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3da80c7bf0 con 0x7f3da815abe0 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f7a9b7fe700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7aa006afa0 0x7f7aa006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa8088c70 con 0x7f7a9c0042d0 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f7a9b7fe700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7f7aa0073140 0x7f7aa0075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 --> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7aa0075c10 con 0x7f7aa0073140 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f7ab37fe700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7f7aa0073140 0x7f7aa0075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa8053db0 con 0x7f7a9c0042d0 2023-11-24T07:58:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30680094a0 con 0x7f30780b3b50 2023-11-24T07:58:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307815a5e0 con 0x7f30780b3b50 2023-11-24T07:58:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7f1dc8ca6700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7f1db0073070 0x7f1db0075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.608+0000 7fe5e5b98700 1 -- start start 2023-11-24T07:58:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7fe5e5b98700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e015d6a0 0x7fe5e00b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7fe5e5b98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 0x7fe5e00714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7fe5e5b98700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e0071e70 con 0x7fe5e0071a10 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7fe5e5b98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e0072d60 con 0x7fe5e00b4090 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7fe5e5b98700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e006de80 con 0x7fe5e015d6a0 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7fe5deffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 0x7fe5e00714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7fe5deffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 0x7fe5e00714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56568/0 (socket says 172.21.15.32:56568) 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7fe5deffd700 1 -- 172.21.15.32:0/1717114570 learned_addr learned my addr 172.21.15.32:0/1717114570 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.609+0000 7efd42edd700 1 -- start start 2023-11-24T07:58:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7fe5df7fe700 1 --2- 172.21.15.32:0/1717114570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e015d6a0 0x7fe5e00b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f7ab27fc700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7aa006afa0 0x7f7aa006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3078165bf0 con 0x7f30780b3b50 2023-11-24T07:58:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f1dc37fe700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1db006aed0 0x7f1db006d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40729b0 tx=0x7f1db8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd42edd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 0x7efd340eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd42edd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd34057c40 0x7efd340580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd42edd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd34054560 con 0x7efd34054010 2023-11-24T07:58:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd42edd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd34057160 con 0x7efd34057c40 2023-11-24T07:58:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd42edd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd34057440 con 0x7efd340eaab0 2023-11-24T07:58:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3da0014930 con 0x7f3da815abe0 2023-11-24T07:58:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f1dc8ca6700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7f1db0073070 0x7f1db0075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f7ab37fe700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7f7aa0073140 0x7f7aa0075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7fe5df7fe700 1 -- 172.21.15.32:0/1717114570 >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 msgr2=0x7fe5e00714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7fe5df7fe700 1 --2- 172.21.15.32:0/1717114570 >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 0x7fe5e00714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7fe5df7fe700 1 -- 172.21.15.32:0/1717114570 >> v1:172.21.15.123:6789/0 conn(0x7fe5e0071a10 legacy=0x7fe5e006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7fe5df7fe700 1 -- 172.21.15.32:0/1717114570 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00720e0 con 0x7fe5e015d6a0 2023-11-24T07:58:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.610+0000 7f25f81f9700 1 -- start start 2023-11-24T07:58:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd41edb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 0x7efd340eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd41edb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 0x7efd340eb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51826/0 (socket says 172.21.15.32:51826) 2023-11-24T07:58:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7efd41edb700 1 -- 172.21.15.32:0/94272101 learned_addr learned my addr 172.21.15.32:0/94272101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f81f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f00af6d0 0x7f25f0164010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f81f9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f0162220 0x7f25f0164550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f81f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 0x7f25f015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f81f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0163b60 con 0x7f25f00af6d0 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f81f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f014c760 con 0x7f25f0166660 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f81f9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0162f50 con 0x7f25f0162220 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f3d9a7fc700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3d8c06afa0 0x7f3d8c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3da0089d40 con 0x7f3da815abe0 2023-11-24T07:58:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7fe5deffd700 1 --2- 172.21.15.32:0/1717114570 >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 0x7fe5e00714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f5794700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f0162220 0x7f25f0164550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f5794700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f0162220 0x7f25f0164550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51830/0 (socket says 172.21.15.32:51830) 2023-11-24T07:58:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f25f5794700 1 -- 172.21.15.32:0/3921034488 learned_addr learned my addr 172.21.15.32:0/3921034488 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f00af6d0 0x7f25f0164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f3d9a7fc700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f3d8c0731c0 0x7f3d8c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 --> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d8c075c90 con 0x7f3d8c0731c0 2023-11-24T07:58:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.611+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da008e030 con 0x7f3da815abe0 2023-11-24T07:58:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f3dad911700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3d8c06afa0 0x7f3d8c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7fe5df7fe700 1 --2- 172.21.15.32:0/1717114570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e015d6a0 0x7fe5e00b3b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009840 tx=0x7fe5d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faed4011f066f16 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7efd41edb700 1 -- 172.21.15.32:0/94272101 >> v2:172.21.15.123:3300/0 conn(0x7efd34057c40 msgr2=0x7efd340580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7efd41edb700 1 --2- 172.21.15.32:0/94272101 >> v2:172.21.15.123:3300/0 conn(0x7efd34057c40 0x7efd340580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7efd41edb700 1 -- 172.21.15.32:0/94272101 >> v1:172.21.15.123:6789/0 conn(0x7efd34054010 legacy=0x7efd340eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7efd41edb700 1 -- 172.21.15.32:0/94272101 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34057720 con 0x7efd340eaab0 2023-11-24T07:58:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f3dad110700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f3d8c0731c0 0x7f3d8c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7fe5de7fc700 1 -- 172.21.15.32:0/1717114570 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0011830 con 0x7fe5e015d6a0 2023-11-24T07:58:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 -- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f0162220 msgr2=0x7f25f0164550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25f0162220 0x7f25f0164550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 -- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 msgr2=0x7f25f015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25f0166660 0x7f25f015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 -- 172.21.15.32:0/3921034488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009550 con 0x7f25f00af6d0 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7fe5de7fc700 1 -- 172.21.15.32:0/1717114570 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0011990 con 0x7fe5e015d6a0 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7fe5de7fc700 1 -- 172.21.15.32:0/1717114570 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0011d20 con 0x7fe5e015d6a0 2023-11-24T07:58:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f3dad110700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f3d8c0731c0 0x7f3d8c0755e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f7ab27fc700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7aa006afa0 0x7f7aa006d460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006ba0 tx=0x7f7aa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.612+0000 7f25f5f95700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f00af6d0 0x7f25f0164010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25e800cbb0 tx=0x7f25e800cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8003c00 con 0x7f25f00af6d0 2023-11-24T07:58:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e8003d60 con 0x7f25f00af6d0 2023-11-24T07:58:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e800ab20 con 0x7f25f00af6d0 2023-11-24T07:58:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3068003bc0 con 0x7f30780b3b50 2023-11-24T07:58:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f3dad911700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3d8c06afa0 0x7f3d8c06d460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009c40 tx=0x7f3d9c011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f0163230 con 0x7f25f00af6d0 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f01637f0 con 0x7f25f00af6d0 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f306effd700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f305c06afa0 0x7f305c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f306808abe0 con 0x7f30780b3b50 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f307e575700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f305c06afa0 0x7f305c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f306effd700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f305c0731c0 0x7f305c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 --> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f305c075c90 con 0x7f305c0731c0 2023-11-24T07:58:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f306808af90 con 0x7f30780b3b50 2023-11-24T07:58:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== osd.4 v2:172.21.15.123:6808/1395851986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7aa0075c10 con 0x7f7aa0073140 2023-11-24T07:58:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7f307ed76700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f305c0731c0 0x7f305c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== osd.6 v2:172.21.15.123:6800/2373254053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1db0075b40 con 0x7f1db0073070 2023-11-24T07:58:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7f307e575700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f305c06afa0 0x7f305c06d460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3074009c10 tx=0x7f3074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7fe5e5b98700 1 -- 172.21.15.32:0/1717114570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e015d6a0 msgr2=0x7fe5e00b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.613+0000 7fe5e5b98700 1 --2- 172.21.15.32:0/1717114570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e015d6a0 0x7fe5e00b3b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009840 tx=0x7fe5d0009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25d8000fb0 con 0x7f25f00af6d0 2023-11-24T07:58:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7efd41edb700 1 --2- 172.21.15.32:0/94272101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 0x7efd340eb1b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd380098f0 tx=0x7efd38009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f061934dea994741 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7fe5e5b98700 1 -- 172.21.15.32:0/1717114570 shutdown_connections 2023-11-24T07:58:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7fe5e5b98700 1 --2- 172.21.15.32:0/1717114570 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e015d6a0 0x7fe5e00b3b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7fe5e5b98700 1 --2- 172.21.15.32:0/1717114570 >> v2:172.21.15.32:3300/0 conn(0x7fe5e00b4090 0x7fe5e00714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7fe5e5b98700 1 -- 172.21.15.32:0/1717114570 >> 172.21.15.32:0/1717114570 conn(0x7fe5e00cfa30 msgr2=0x7fe5e00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7f307ed76700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f305c0731c0 0x7f305c0755e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7fe5e5b98700 1 -- 172.21.15.32:0/1717114570 shutdown_connections 2023-11-24T07:58:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.614+0000 7fe5e5b98700 1 -- 172.21.15.32:0/1717114570 wait complete. 2023-11-24T07:58:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7efd40ed9700 1 -- 172.21.15.32:0/94272101 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38004230 con 0x7efd340eaab0 2023-11-24T07:58:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7efd40ed9700 1 -- 172.21.15.32:0/94272101 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38004390 con 0x7efd340eaab0 2023-11-24T07:58:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7efd40ed9700 1 -- 172.21.15.32:0/94272101 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380036a0 con 0x7efd340eaab0 2023-11-24T07:58:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== osd.5 v2:172.21.15.123:6820/3666097374 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f305c075c90 con 0x7f305c0731c0 2023-11-24T07:58:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7efd42edd700 1 -- 172.21.15.32:0/94272101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 msgr2=0x7efd340eb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7efd42edd700 1 --2- 172.21.15.32:0/94272101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 0x7efd340eb1b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd380098f0 tx=0x7efd38009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.615+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25e800ac80 con 0x7f25f00af6d0 2023-11-24T07:58:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== osd.3 v2:172.21.15.32:6808/2834432589 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3d8c075c90 con 0x7f3d8c0731c0 2023-11-24T07:58:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7fe5e5b98700 1 Processor -- start 2023-11-24T07:58:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7efd42edd700 1 -- 172.21.15.32:0/94272101 shutdown_connections 2023-11-24T07:58:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7efd42edd700 1 --2- 172.21.15.32:0/94272101 >> v2:172.21.15.123:3300/0 conn(0x7efd34057c40 0x7efd340580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7efd42edd700 1 --2- 172.21.15.32:0/94272101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd340eaab0 0x7efd340eb1b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7efd42edd700 1 -- 172.21.15.32:0/94272101 >> 172.21.15.32:0/94272101 conn(0x7efd34055a30 msgr2=0x7efd34055e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7fed73869700 1 Processor -- start 2023-11-24T07:58:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7fe5e5b98700 1 -- start start 2023-11-24T07:58:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fe5e5b98700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e0071a10 0x7fe5e0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fe5e5b98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e00b4090 0x7fe5e01f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fe5e5b98700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e015d6a0 0x7fe5e01ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fe5e5b98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e01f3f00 con 0x7fe5e00b4090 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fe5e5b98700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00a9810 con 0x7fe5e015d6a0 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fe5e5b98700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0157020 con 0x7fe5e0071a10 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7fed73869700 1 -- start start 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fed73869700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7fed73869700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed6c06e930 0x7fed6c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed73869700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c06f9e0 con 0x7fed6c15ece0 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed73869700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c15f150 con 0x7fed6c06e930 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed73869700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c06edd0 con 0x7fed6c144590 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f25e67fc700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f25d406aed0 0x7f25d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25e80895d0 con 0x7f25f00af6d0 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f25e67fc700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f25d4073070 0x7f25d4075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 --> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25d4075b40 con 0x7f25d4073070 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25e808bc90 con 0x7f25f00af6d0 2023-11-24T07:58:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.616+0000 7f25f6796700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f25d4073070 0x7f25d4075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.617+0000 7f25f6796700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f25d4073070 0x7f25d4075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e00b4090 0x7fe5e01f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71e06700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed6c15ece0 0x7fed6c15b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58316/0 (socket says 172.21.15.32:58316) 2023-11-24T07:58:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71e06700 1 -- 172.21.15.32:0/3130262001 learned_addr learned my addr 172.21.15.32:0/3130262001 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71605700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e00b4090 0x7fe5e01f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56596/0 (socket says 172.21.15.32:56596) 2023-11-24T07:58:44.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 -- 172.21.15.32:0/3530076506 learned_addr learned my addr 172.21.15.32:0/3530076506 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5dffff700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e015d6a0 0x7fe5e01ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5df7fe700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e0071a10 0x7fe5e0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7f25f5794700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f25d406aed0 0x7f25d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 -- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e0071a10 msgr2=0x7fe5e0156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5e0071a10 0x7fe5e0156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 -- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e015d6a0 msgr2=0x7fe5e01ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e015d6a0 0x7fe5e01ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fe5deffd700 1 -- 172.21.15.32:0/3530076506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c80085d0 con 0x7fe5e00b4090 2023-11-24T07:58:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7f25f5794700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f25d406aed0 0x7f25d406d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25f014f470 tx=0x7f25ec008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fe5deffd700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e00b4090 0x7fe5e01f39c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8010fb0 tx=0x7fe5c800e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71605700 1 -- 172.21.15.32:0/3130262001 >> v2:172.21.15.123:3300/0 conn(0x7fed6c06e930 msgr2=0x7fed6c15e7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71605700 1 --2- 172.21.15.32:0/3130262001 >> v2:172.21.15.123:3300/0 conn(0x7fed6c06e930 0x7fed6c15e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71605700 1 -- 172.21.15.32:0/3130262001 >> v1:172.21.15.32:6789/0 conn(0x7fed6c15ece0 legacy=0x7fed6c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed63fff700 1 -- 172.21.15.32:0/3130262001 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266101460 0 0) 0x7fed6c06f9e0 con 0x7fed6c15ece0 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.618+0000 7fed71605700 1 -- 172.21.15.32:0/3130262001 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c06f0b0 con 0x7fed6c144590 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fed71605700 1 --2- 172.21.15.32:0/3130262001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c06e3f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed64009d90 tx=0x7fed6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a320e620fd481d0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fed63fff700 1 -- 172.21.15.32:0/3130262001 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400e070 con 0x7fed6c144590 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fed63fff700 1 -- 172.21.15.32:0/3130262001 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6400bbd0 con 0x7fed6c144590 2023-11-24T07:58:44.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fed63fff700 1 -- 172.21.15.32:0/3130262001 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64012430 con 0x7fed6c144590 2023-11-24T07:58:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800eee0 con 0x7fe5e00b4090 2023-11-24T07:58:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8011040 con 0x7fe5e00b4090 2023-11-24T07:58:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d00094a0 con 0x7fe5e00b4090 2023-11-24T07:58:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e01575e0 con 0x7fe5e00b4090 2023-11-24T07:58:44.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.619+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004250 con 0x7fe5e00b4090 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fed73869700 1 -- 172.21.15.32:0/3130262001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 msgr2=0x7fed6c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fed73869700 1 --2- 172.21.15.32:0/3130262001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c06e3f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed64009d90 tx=0x7fed6400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fed73869700 1 -- 172.21.15.32:0/3130262001 shutdown_connections 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fed73869700 1 --2- 172.21.15.32:0/3130262001 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c06e3f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fed73869700 1 --2- 172.21.15.32:0/3130262001 >> v2:172.21.15.123:3300/0 conn(0x7fed6c06e930 0x7fed6c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fed73869700 1 -- 172.21.15.32:0/3130262001 >> 172.21.15.32:0/3130262001 conn(0x7fed6c0d0ba0 msgr2=0x7fed6c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.620+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5c4000fb0 con 0x7fe5e00b4090 2023-11-24T07:58:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7efd42edd700 1 -- 172.21.15.32:0/94272101 shutdown_connections 2023-11-24T07:58:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7fed73869700 1 -- 172.21.15.32:0/3130262001 shutdown_connections 2023-11-24T07:58:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5c80043b0 con 0x7fe5e00b4090 2023-11-24T07:58:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== osd.0 v2:172.21.15.32:6816/843272889 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f25d4075b40 con 0x7f25d4073070 2023-11-24T07:58:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7fe5d7fff700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5c006aed0 0x7fe5c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.622+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5c808c340 con 0x7fe5e00b4090 2023-11-24T07:58:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.622+0000 7fe5d7fff700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7fe5c0073070 0x7fe5c0075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.622+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 --> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5c0075b40 con 0x7fe5c0073070 2023-11-24T07:58:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7efd42edd700 1 -- 172.21.15.32:0/94272101 wait complete. 2023-11-24T07:58:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7efd42edd700 1 Processor -- start 2023-11-24T07:58:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.621+0000 7fed73869700 1 -- 172.21.15.32:0/3130262001 wait complete. 2023-11-24T07:58:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7fed73869700 1 Processor -- start 2023-11-24T07:58:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7fe5dffff700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7fe5c0073070 0x7fe5c0075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7efd42edd700 1 -- start start 2023-11-24T07:58:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c80574a0 con 0x7fe5e00b4090 2023-11-24T07:58:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7fe5dffff700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7fe5c0073070 0x7fe5c0075490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd42edd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd34054010 0x7efd3404f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd42edd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd34057c40 0x7efd3403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd42edd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd340eaab0 0x7efd34039800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd42edd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34039370 con 0x7efd34057c40 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd42edd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34038ab0 con 0x7efd340eaab0 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd42edd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd34038d20 con 0x7efd34054010 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd41edb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd34054010 0x7efd3404f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd41edb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd34054010 0x7efd3404f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51864/0 (socket says 172.21.15.32:51864) 2023-11-24T07:58:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd416da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd34057c40 0x7efd3403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd41edb700 1 -- 172.21.15.32:0/1442321332 learned_addr learned my addr 172.21.15.32:0/1442321332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd426dc700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd340eaab0 0x7efd34039800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd426dc700 1 -- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd34054010 msgr2=0x7efd3404f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd426dc700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd34054010 0x7efd3404f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== osd.7 v2:172.21.15.123:6816/3074243597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe5c0075b40 con 0x7fe5c0073070 2023-11-24T07:58:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7efd426dc700 1 -- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd34057c40 msgr2=0x7efd3403ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7efd426dc700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd34057c40 0x7efd3403ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7efd426dc700 1 -- 172.21.15.32:0/1442321332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38009550 con 0x7efd340eaab0 2023-11-24T07:58:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7efd416da700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd34057c40 0x7efd3403ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.623+0000 7fed73869700 1 -- start start 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fed73869700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c200190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fed73869700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6c15ece0 0x7fed6c13ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fed73869700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed6c0b07c0 0x7fed6c13c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fed73869700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c13b380 con 0x7fed6c15ece0 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fed73869700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c13f200 con 0x7fed6c0b07c0 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.624+0000 7fed73869700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0b0f60 con 0x7fed6c144590 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c200190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c200190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51870/0 (socket says 172.21.15.32:51870) 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 -- 172.21.15.32:0/1140823139 learned_addr learned my addr 172.21.15.32:0/1140823139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed70e04700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6c15ece0 0x7fed6c13ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 -- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed6c0b07c0 msgr2=0x7fed6c13c780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed6c0b07c0 0x7fed6c13c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 -- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6c15ece0 msgr2=0x7fed6c13ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6c15ece0 0x7fed6c13ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 -- 172.21.15.32:0/1140823139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64009a70 con 0x7fed6c144590 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed70e04700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6c15ece0 0x7fed6c13ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.625+0000 7fed71605700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c200190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b730 tx=0x7fed64004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.626+0000 7fe5df7fe700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5c006aed0 0x7fe5c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.626+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64003ea0 con 0x7fed6c144590 2023-11-24T07:58:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.626+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed64020070 con 0x7fed6c144590 2023-11-24T07:58:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.626+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64004280 con 0x7fed6c144590 2023-11-24T07:58:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.626+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0b1240 con 0x7fed6c144590 2023-11-24T07:58:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.626+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c0b0c50 con 0x7fed6c144590 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.627+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed6c13f080 con 0x7fed6c144590 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.628+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed640043e0 con 0x7fed6c144590 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed61ffb700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fed5006af20 0x7fed5006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed64022070 con 0x7fed6c144590 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed61ffb700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fed50073140 0x7fed50075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 --> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed50075c10 con 0x7fed50073140 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed6408c200 con 0x7fed6c144590 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed70e04700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fed5006af20 0x7fed5006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.629+0000 7fed71e06700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fed50073140 0x7fed50075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.630+0000 7fed70e04700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fed5006af20 0x7fed5006d3e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0b5190 tx=0x7fed68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.630+0000 7fed71e06700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fed50073140 0x7fed50075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.631+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== osd.2 v2:172.21.15.32:6824/3901267389 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fed50075c10 con 0x7fed50073140 2023-11-24T07:58:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.637+0000 7fe5df7fe700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5c006aed0 0x7fe5c006d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009fd0 tx=0x7fe5d001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.638+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 --> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1dc41f3700 con 0x7f1db0073070 2023-11-24T07:58:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.638+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 --> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3da80ce7b0 con 0x7f3d8c0731c0 2023-11-24T07:58:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7efd426dc700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd340eaab0 0x7efd34039800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd3c050c30 tx=0x7efd3c075550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7f3d9a7fc700 1 -- 172.21.15.32:0/3579549995 <== osd.3 v2:172.21.15.32:6808/2834432589 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3da4001840 con 0x7f3d8c0731c0 2023-11-24T07:58:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c04f7b0 con 0x7efd340eaab0 2023-11-24T07:58:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c073470 con 0x7efd340eaab0 2023-11-24T07:58:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0933b0 con 0x7efd340eaab0 2023-11-24T07:58:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd34039000 con 0x7efd340eaab0 2023-11-24T07:58:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.639+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3417e7d0 con 0x7efd340eaab0 2023-11-24T07:58:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.640+0000 7f1dc0ff9700 1 -- 172.21.15.32:0/875210696 <== osd.6 v2:172.21.15.123:6800/2373254053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1db4001610 con 0x7f1db0073070 2023-11-24T07:58:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.640+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7f1db0073070 msgr2=0x7f1db0075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.640+0000 7f1dca709700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7f1db0073070 0x7f1db0075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.640+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1db006aed0 msgr2=0x7f1db006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.640+0000 7f1dca709700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1db006aed0 0x7f1db006d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40729b0 tx=0x7f1db8008040 comp rx=0 tx=0).stop 2023-11-24T07:58:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.641+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac003a50 msgr2=0x7f1dc41f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.641+0000 7f1dca709700 1 --2- 172.21.15.32:0/875210696 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1dac003a50 0x7f1dc41f3a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc019000 tx=0x7f1dbc0013d0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.641+0000 7f1dc8ca6700 1 -- 172.21.15.32:0/875210696 reap_dead start 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.641+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 shutdown_connections 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.641+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 >> 172.21.15.32:0/875210696 conn(0x7f1dc406cec0 msgr2=0x7f1dc415a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.643+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 shutdown_connections 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.643+0000 7f1dca709700 1 -- 172.21.15.32:0/875210696 wait complete. 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.645+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 --> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ab40ce830 con 0x7f7aa0073140 2023-11-24T07:58:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/373919222 <== osd.4 v2:172.21.15.123:6808/1395851986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7ab40ce830 con 0x7f7aa0073140 2023-11-24T07:58:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7f7aa0073140 msgr2=0x7f7aa0075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab96ee700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7f7aa0073140 0x7f7aa0075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7aa006afa0 msgr2=0x7f7aa006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab96ee700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7aa006afa0 0x7f7aa006d460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006ba0 tx=0x7f7aa4008040 comp rx=0 tx=0).stop 2023-11-24T07:58:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 msgr2=0x7f7ab413d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab96ee700 1 --2- 172.21.15.32:0/373919222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a9c0042d0 0x7f7ab413d1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800e580 tx=0x7f7aa800e890 comp rx=0 tx=0).stop 2023-11-24T07:58:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.646+0000 7f7ab37fe700 1 -- 172.21.15.32:0/373919222 reap_dead start 2023-11-24T07:58:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f3d8c0731c0 msgr2=0x7f3d8c0755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f3d8c0731c0 0x7f3d8c0755e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3d8c06afa0 msgr2=0x7f3d8c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3d8c06afa0 0x7f3d8c06d460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009c40 tx=0x7f3d9c011000 comp rx=0 tx=0).stop 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da815abe0 msgr2=0x7f3da80cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 --2- 172.21.15.32:0/3579549995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3da815abe0 0x7f3da80cbad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3da0007e40 tx=0x7f3da000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dae112700 1 -- 172.21.15.32:0/3579549995 reap_dead start 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 shutdown_connections 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.647+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 >> 172.21.15.32:0/3579549995 conn(0x7f3da8072610 msgr2=0x7f3da806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.648+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd3403b930 con 0x7efd340eaab0 2023-11-24T07:58:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.651+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 shutdown_connections 2023-11-24T07:58:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.651+0000 7f3dafb75700 1 -- 172.21.15.32:0/3579549995 wait complete. 2023-11-24T07:58:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.651+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 --> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe5c4002d30 con 0x7fe5c0073070 2023-11-24T07:58:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.651+0000 7fe5d7fff700 1 -- 172.21.15.32:0/3530076506 <== osd.7 v2:172.21.15.123:6816/3074243597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe5c4002d30 con 0x7fe5c0073070 2023-11-24T07:58:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.652+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7fe5c0073070 msgr2=0x7fe5c0075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.652+0000 7fe5e5b98700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7fe5c0073070 0x7fe5c0075490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd3c093510 con 0x7efd340eaab0 2023-11-24T07:58:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 shutdown_connections 2023-11-24T07:58:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 >> 172.21.15.32:0/373919222 conn(0x7f7ab40cfa30 msgr2=0x7f7ab4140610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7efd327fc700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efd2006aec0 0x7efd2006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd3c103df0 con 0x7efd340eaab0 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7efd41edb700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efd2006aec0 0x7efd2006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.654+0000 7efd327fc700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7efd20073060 0x7efd20075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.654+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 --> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd20075b30 con 0x7efd20073060 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.654+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd3c1064b0 con 0x7efd340eaab0 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.654+0000 7efd41edb700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efd2006aec0 0x7efd2006d380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd38007fd0 tx=0x7efd38008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.655+0000 7efd416da700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7efd20073060 0x7efd20075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.655+0000 7efd416da700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7efd20073060 0x7efd20075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.696 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:44.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T07:58:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.657+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 shutdown_connections 2023-11-24T07:58:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.657+0000 7f7ab96ee700 1 -- 172.21.15.32:0/373919222 wait complete. 2023-11-24T07:58:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.657+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== osd.1 v2:172.21.15.32:6800/835572876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7efd20075b30 con 0x7efd20073060 2023-11-24T07:58:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.658+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 --> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed6c06f670 con 0x7fed50073140 2023-11-24T07:58:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.658+0000 7fed61ffb700 1 -- 172.21.15.32:0/1140823139 <== osd.2 v2:172.21.15.32:6824/3901267389 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed5c001150 con 0x7fed50073140 2023-11-24T07:58:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fed50073140 msgr2=0x7fed50075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed73869700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fed50073140 0x7fed50075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fed5006af20 msgr2=0x7fed5006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed73869700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fed5006af20 0x7fed5006d3e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0b5190 tx=0x7fed68008040 comp rx=0 tx=0).stop 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 msgr2=0x7fed6c200190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed73869700 1 --2- 172.21.15.32:0/1140823139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed6c144590 0x7fed6c200190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b730 tx=0x7fed64004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.659+0000 7fed71e06700 1 -- 172.21.15.32:0/1140823139 reap_dead start 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5c006aed0 msgr2=0x7fe5c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5e5b98700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5c006aed0 0x7fe5c006d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009fd0 tx=0x7fe5d001c040 comp rx=0 tx=0).stop 2023-11-24T07:58:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e00b4090 msgr2=0x7fe5e01f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5e5b98700 1 --2- 172.21.15.32:0/3530076506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e00b4090 0x7fe5e01f39c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8010fb0 tx=0x7fe5c800e3a0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5dffff700 1 -- 172.21.15.32:0/3530076506 reap_dead start 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 shutdown_connections 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.653+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 >> 172.21.15.32:0/3530076506 conn(0x7fe5e00cfa30 msgr2=0x7fe5e00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.660+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 --> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f307806cb60 con 0x7f305c0731c0 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.660+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 shutdown_connections 2023-11-24T07:58:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.660+0000 7fe5e5b98700 1 -- 172.21.15.32:0/3530076506 wait complete. 2023-11-24T07:58:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.661+0000 7f306effd700 1 -- 172.21.15.32:0/1924094833 <== osd.5 v2:172.21.15.123:6820/3666097374 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3070001fc0 con 0x7f305c0731c0 2023-11-24T07:58:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f305c0731c0 msgr2=0x7f305c0755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f305c0731c0 0x7f305c0755e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f305c06afa0 msgr2=0x7f305c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f305c06afa0 0x7f305c06d460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3074009c10 tx=0x7f3074008040 comp rx=0 tx=0).stop 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30780b3b50 msgr2=0x7f30781593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 --2- 172.21.15.32:0/1924094833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30780b3b50 0x7f30781593d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3068009fd0 tx=0x7f3068004750 comp rx=0 tx=0).stop 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f307ed76700 1 -- 172.21.15.32:0/1924094833 reap_dead start 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 shutdown_connections 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 >> 172.21.15.32:0/1924094833 conn(0x7f30780cfa30 msgr2=0x7f30780710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 shutdown_connections 2023-11-24T07:58:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.662+0000 7f30807d9700 1 -- 172.21.15.32:0/1924094833 wait complete. 2023-11-24T07:58:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.667+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 --> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25d8002d30 con 0x7f25d4073070 2023-11-24T07:58:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.668+0000 7f25e67fc700 1 -- 172.21.15.32:0/3921034488 <== osd.0 v2:172.21.15.32:6816/843272889 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25d8002d30 con 0x7f25d4073070 2023-11-24T07:58:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.669+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 shutdown_connections 2023-11-24T07:58:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.669+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 >> 172.21.15.32:0/1140823139 conn(0x7fed6c0d0ba0 msgr2=0x7fed6c15b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:44.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T07:58:44.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.671+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 shutdown_connections 2023-11-24T07:58:44.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.671+0000 7fed73869700 1 -- 172.21.15.32:0/1140823139 wait complete. 2023-11-24T07:58:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:44.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T07:58:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.679+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f25d4073070 msgr2=0x7f25d4075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.679+0000 7f25f81f9700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f25d4073070 0x7f25d4075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:44.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T07:58:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f25d406aed0 msgr2=0x7f25d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f25d406aed0 0x7f25d406d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25f014f470 tx=0x7f25ec008030 comp rx=0 tx=0).stop 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f00af6d0 msgr2=0x7f25f0164010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 --2- 172.21.15.32:0/3921034488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f00af6d0 0x7f25f0164010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25e800cbb0 tx=0x7f25e800cec0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f6796700 1 -- 172.21.15.32:0/3921034488 reap_dead start 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 shutdown_connections 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 >> 172.21.15.32:0/3921034488 conn(0x7f25f006cec0 msgr2=0x7f25f014d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 shutdown_connections 2023-11-24T07:58:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.681+0000 7f25f81f9700 1 -- 172.21.15.32:0/3921034488 wait complete. 2023-11-24T07:58:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-11-24T07:58:44.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T07:58:44.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.685+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 --> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd340e62b0 con 0x7efd20073060 2023-11-24T07:58:44.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd327fc700 1 -- 172.21.15.32:0/1442321332 <== osd.1 v2:172.21.15.32:6800/835572876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd340e62b0 con 0x7efd20073060 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7efd20073060 msgr2=0x7efd20075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd42edd700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7efd20073060 0x7efd20075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efd2006aec0 msgr2=0x7efd2006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd42edd700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efd2006aec0 0x7efd2006d380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd38007fd0 tx=0x7efd38008030 comp rx=0 tx=0).stop 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd340eaab0 msgr2=0x7efd34039800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.686+0000 7efd42edd700 1 --2- 172.21.15.32:0/1442321332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd340eaab0 0x7efd34039800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd3c050c30 tx=0x7efd3c075550 comp rx=0 tx=0).stop 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.687+0000 7efd426dc700 1 -- 172.21.15.32:0/1442321332 reap_dead start 2023-11-24T07:58:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.687+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 shutdown_connections 2023-11-24T07:58:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.687+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 >> 172.21.15.32:0/1442321332 conn(0x7efd34055a30 msgr2=0x7efd340e5310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.687+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 shutdown_connections 2023-11-24T07:58:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.687+0000 7efd42edd700 1 -- 172.21.15.32:0/1442321332 wait complete. 2023-11-24T07:58:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:44.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T07:58:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:44.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T07:58:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:44.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T07:58:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.964+0000 7f46819d7700 1 Processor -- start 2023-11-24T07:58:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.964+0000 7f46819d7700 1 -- start start 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f46819d7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c06d4d0 0x7f467c16cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f46819d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c06ced0 con 0x7f467c0b3b50 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f46819d7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c06da70 con 0x7f467c164a90 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f46819d7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1634e0 con 0x7f467c06d4d0 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f467affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f467c0b3b50 0x7f467c0b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58344/0 (socket says 172.21.15.32:58344) 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f467affd700 1 -- 172.21.15.32:0/134353196 learned_addr learned my addr 172.21.15.32:0/134353196 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7f467b7fe700 1 --2- 172.21.15.32:0/134353196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c06d4d0 0x7f467c16cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7fab1ac50700 1 Processor -- start 2023-11-24T07:58:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7f467b7fe700 1 -- 172.21.15.32:0/134353196 >> v1:172.21.15.123:6789/0 conn(0x7f467c164a90 legacy=0x7f467c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7f467b7fe700 1 -- 172.21.15.32:0/134353196 >> v1:172.21.15.32:6789/0 conn(0x7f467c0b3b50 legacy=0x7f467c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7f467b7fe700 1 -- 172.21.15.32:0/134353196 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c1637c0 con 0x7f467c06d4d0 2023-11-24T07:58:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7f467b7fe700 1 --2- 172.21.15.32:0/134353196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c06d4d0 0x7f467c16cef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466c009870 tx=0x7f466c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d9af3db39ad412 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7fab1ac50700 1 -- start start 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.965+0000 7f5f27f27700 1 Processor -- start 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.966+0000 7f5f27f27700 1 -- start start 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f27f27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 0x7f5f2006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f27f27700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f20072bb0 0x7f5f200b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f27f27700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2008bdb0 con 0x7f5f200b4090 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f27f27700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200b4510 con 0x7f5f20072bb0 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f27f27700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2006ce30 con 0x7f5f2016ae60 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 0x7f5f2006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 0x7f5f2006efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51890/0 (socket says 172.21.15.32:51890) 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 -- 172.21.15.32:0/1393914407 learned_addr learned my addr 172.21.15.32:0/1393914407 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f254c2700 1 --2- 172.21.15.32:0/1393914407 >> v2:172.21.15.123:3300/0 conn(0x7f5f20072bb0 0x7f5f200b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f4679ffb700 1 -- 172.21.15.32:0/134353196 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c004230 con 0x7f467c06d4d0 2023-11-24T07:58:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f4679ffb700 1 -- 172.21.15.32:0/134353196 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c004390 con 0x7f467c06d4d0 2023-11-24T07:58:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f4679ffb700 1 -- 172.21.15.32:0/134353196 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c0036a0 con 0x7f467c06d4d0 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 -- 172.21.15.32:0/1393914407 >> v2:172.21.15.123:3300/0 conn(0x7f5f20072bb0 msgr2=0x7f5f200b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 --2- 172.21.15.32:0/1393914407 >> v2:172.21.15.123:3300/0 conn(0x7f5f20072bb0 0x7f5f200b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 -- 172.21.15.32:0/1393914407 >> v1:172.21.15.123:6789/0 conn(0x7f5f200b4090 legacy=0x7f5f2016e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.967+0000 7f5f25cc3700 1 -- 172.21.15.32:0/1393914407 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2006d110 con 0x7f5f2016ae60 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f5f25cc3700 1 --2- 172.21.15.32:0/1393914407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 0x7f5f2006efc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009e50 tx=0x7f5f1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993ecbda6e7a2565 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f5f24cc1700 1 -- 172.21.15.32:0/1393914407 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800e070 con 0x7f5f2016ae60 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f5f24cc1700 1 -- 172.21.15.32:0/1393914407 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18003a20 con 0x7f5f2016ae60 2023-11-24T07:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f5f24cc1700 1 -- 172.21.15.32:0/1393914407 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003dd0 con 0x7f5f2016ae60 2023-11-24T07:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f46819d7700 1 -- 172.21.15.32:0/134353196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c06d4d0 msgr2=0x7f467c16cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f46819d7700 1 --2- 172.21.15.32:0/134353196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c06d4d0 0x7f467c16cef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466c009870 tx=0x7f466c009b80 comp rx=0 tx=0).stop 2023-11-24T07:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f46819d7700 1 -- 172.21.15.32:0/134353196 shutdown_connections 2023-11-24T07:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f46819d7700 1 --2- 172.21.15.32:0/134353196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c06d4d0 0x7f467c16cef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7f46819d7700 1 -- 172.21.15.32:0/134353196 >> 172.21.15.32:0/134353196 conn(0x7f467c0cfa30 msgr2=0x7f467c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f9f5b022700 1 Processor -- start 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f9f5b022700 1 -- start start 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab1ac50700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab140b4260 0x7fab140714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab1ac50700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fab1415d6a0 0x7fab14071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab1ac50700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab140720e0 con 0x7fab140d0520 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab1ac50700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab14072d60 con 0x7fab1415d6a0 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab1ac50700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1406de80 con 0x7fab140b4260 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab189ec700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab140d0520 0x7fab140b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58356/0 (socket says 172.21.15.32:58356) 2023-11-24T07:58:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.968+0000 7fab189ec700 1 -- 172.21.15.32:0/1924308822 learned_addr learned my addr 172.21.15.32:0/1924308822 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f5f27f27700 1 -- 172.21.15.32:0/1393914407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 msgr2=0x7f5f2006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f5f27f27700 1 --2- 172.21.15.32:0/1393914407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 0x7f5f2006efc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009e50 tx=0x7f5f1800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128639344 0 0) 0x7fab140720e0 con 0x7fab140d0520 2023-11-24T07:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faafc0035b0 con 0x7fab140d0520 2023-11-24T07:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab191ed700 1 --2- 172.21.15.32:0/1924308822 >> v2:172.21.15.123:3300/0 conn(0x7fab1415d6a0 0x7fab14071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab13fff700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab140b4260 0x7fab140714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f5f27f27700 1 -- 172.21.15.32:0/1393914407 shutdown_connections 2023-11-24T07:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f5f27f27700 1 --2- 172.21.15.32:0/1393914407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f2016ae60 0x7f5f2006efc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f5f27f27700 1 --2- 172.21.15.32:0/1393914407 >> v2:172.21.15.123:3300/0 conn(0x7f5f20072bb0 0x7f5f200b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7f5f27f27700 1 -- 172.21.15.32:0/1393914407 >> 172.21.15.32:0/1393914407 conn(0x7f5f200cfa30 msgr2=0x7f5f200cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f46819d7700 1 -- 172.21.15.32:0/134353196 shutdown_connections 2023-11-24T07:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f5f27f27700 1 -- 172.21.15.32:0/1393914407 shutdown_connections 2023-11-24T07:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fd1186b8700 1 Processor -- start 2023-11-24T07:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1527053626 0 0) 0x7faafc0035b0 con 0x7fab140d0520 2023-11-24T07:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab140b4260 msgr2=0x7fab140714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fab140b4260 0x7fab140714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 >> v2:172.21.15.123:3300/0 conn(0x7fab1415d6a0 msgr2=0x7fab14071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 --2- 172.21.15.32:0/1924308822 >> v2:172.21.15.123:3300/0 conn(0x7fab1415d6a0 0x7fab14071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140723c0 con 0x7fab140d0520 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.969+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fab04001670 con 0x7fab140d0520 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 >> v1:172.21.15.32:6789/0 conn(0x7fab140d0520 legacy=0x7fab140b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc003a00 0x7faafc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc0043f0 0x7faafc0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc0050d0 0x7faafc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14072d60 con 0x7faafc0050d0 2023-11-24T07:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1406de80 con 0x7faafc0043f0 2023-11-24T07:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f9f5b022700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f5415ff10 0x7f9f540c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f9f5b022700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f54162a00 con 0x7f9f540cc330 2023-11-24T07:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f9f5b022700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f54162c40 con 0x7f9f540b3370 2023-11-24T07:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f9f5b022700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f54162f20 con 0x7f9f5415ff10 2023-11-24T07:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc00c080 con 0x7faafc003a00 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab191ed700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc0050d0 0x7faafc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab189ec700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc003a00 0x7faafc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab191ed700 1 -- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc003a00 msgr2=0x7faafc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab191ed700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc003a00 0x7faafc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab191ed700 1 -- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc0043f0 msgr2=0x7faafc0048d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab191ed700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc0043f0 0x7faafc0048d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fab191ed700 1 -- 172.21.15.32:0/1924308822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140723c0 con 0x7faafc0050d0 2023-11-24T07:58:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7fab191ed700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc0050d0 0x7faafc00ba10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009f70 tx=0x7faafc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb5e64ec1711a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7f9f53fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9f540cc330 0x7f9f540b2c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58368/0 (socket says 172.21.15.32:58368) 2023-11-24T07:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7f9f53fff700 1 -- 172.21.15.32:0/3704127205 learned_addr learned my addr 172.21.15.32:0/3704127205 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c003a20 con 0x7faafc0050d0 2023-11-24T07:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c00f040 con 0x7faafc0050d0 2023-11-24T07:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7fab137fe700 1 -- 172.21.15.32:0/1924308822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c003d80 con 0x7faafc0050d0 2023-11-24T07:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f46819d7700 1 -- 172.21.15.32:0/134353196 wait complete. 2023-11-24T07:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7f46819d7700 1 Processor -- start 2023-11-24T07:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7fd1186b8700 1 -- start start 2023-11-24T07:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.971+0000 7fa093636700 1 Processor -- start 2023-11-24T07:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fa093636700 1 -- start start 2023-11-24T07:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.970+0000 7f5f27f27700 1 -- 172.21.15.32:0/1393914407 wait complete. 2023-11-24T07:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd1186b8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd110070fd0 0x7fd110165e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd1186b8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1101663d0 0x7fd1100b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd1186b8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd110072110 con 0x7fd1100cf440 2023-11-24T07:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd1186b8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd110166860 con 0x7fd110070fd0 2023-11-24T07:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd1186b8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd110071500 con 0x7fd1101663d0 2023-11-24T07:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7f5f27f27700 1 Processor -- start 2023-11-24T07:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7f46819d7700 1 -- start start 2023-11-24T07:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd115c53700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd110070fd0 0x7fd110165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd115c53700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd110070fd0 0x7fd110165e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40814/0 (socket says 172.21.15.32:40814) 2023-11-24T07:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd115c53700 1 -- 172.21.15.32:0/3429400859 learned_addr learned my addr 172.21.15.32:0/3429400859 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd116c55700 1 --2- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1101663d0 0x7fd1100b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fd116454700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd1100cf440 0x7fd11006e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45128/0 (socket says 172.21.15.32:45128) 2023-11-24T07:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fab1ac50700 1 -- 172.21.15.32:0/1924308822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc0050d0 msgr2=0x7faafc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fab1ac50700 1 --2- 172.21.15.32:0/1924308822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faafc0050d0 0x7faafc00ba10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009f70 tx=0x7faafc005880 comp rx=0 tx=0).stop 2023-11-24T07:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7fab191ed700 1 -- 172.21.15.32:0/1924308822 reap_dead start 2023-11-24T07:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f9f58dbe700 1 --2- 172.21.15.32:0/3704127205 >> v2:172.21.15.32:3300/0 conn(0x7f9f5415ff10 0x7f9f540c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fd115c53700 1 -- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1101663d0 msgr2=0x7fd1100b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fd115c53700 1 --2- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1101663d0 0x7fd1100b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fd115c53700 1 -- 172.21.15.32:0/3429400859 >> v1:172.21.15.123:6789/0 conn(0x7fd1100cf440 legacy=0x7fd11006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fd115c53700 1 -- 172.21.15.32:0/3429400859 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100717e0 con 0x7fd110070fd0 2023-11-24T07:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fab1ac50700 1 -- 172.21.15.32:0/1924308822 shutdown_connections 2023-11-24T07:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fab1ac50700 1 -- 172.21.15.32:0/1924308822 >> 172.21.15.32:0/1924308822 conn(0x7fab140cfa30 msgr2=0x7fab140cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fd115c53700 1 --2- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd110070fd0 0x7fd110165e90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098f0 tx=0x7fd100009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c3bd121e583993 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fa093636700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08c06cac0 0x7fa08c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fa093636700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c071c70 con 0x7fa08c16bf80 2023-11-24T07:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f9f58dbe700 1 -- 172.21.15.32:0/3704127205 >> v1:172.21.15.123:6789/0 conn(0x7f9f540b3370 legacy=0x7f9f540b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f9f58dbe700 1 -- 172.21.15.32:0/3704127205 >> v1:172.21.15.32:6789/0 conn(0x7f9f540cc330 legacy=0x7f9f540b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f9f58dbe700 1 -- 172.21.15.32:0/3704127205 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54163200 con 0x7f9f5415ff10 2023-11-24T07:58:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fa093636700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c06cf40 con 0x7fa08c08b290 2023-11-24T07:58:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7fa093636700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08c0b43a0 con 0x7fa08c06cac0 2023-11-24T07:58:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.972+0000 7f5f27f27700 1 -- start start 2023-11-24T07:58:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f5f27f27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f200b4090 0x7f5f20204910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f5f27f27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 0x7f5f20215700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f46819d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 0x7f467c200000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f46819d7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c0b3b50 0x7f467c200540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f46819d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f467c164a90 0x7f467c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f46819d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c200a80 con 0x7f467c06d4d0 2023-11-24T07:58:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f46819d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0711e0 con 0x7f467c164a90 2023-11-24T07:58:44.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f46819d7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c15ad10 con 0x7f467c0b3b50 2023-11-24T07:58:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f72961e3700 1 Processor -- start 2023-11-24T07:58:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.973+0000 7f72961e3700 1 -- start start 2023-11-24T07:58:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fa0913d2700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa08c08b290 0x7fa08c08b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45134/0 (socket says 172.21.15.32:45134) 2023-11-24T07:58:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fa0913d2700 1 -- 172.21.15.32:0/2520227541 learned_addr learned my addr 172.21.15.32:0/2520227541 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:44.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f9f58dbe700 1 --2- 172.21.15.32:0/3704127205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f540c4cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009840 tx=0x7f9f48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33471b8335a2f669 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fa090bd1700 1 --2- 172.21.15.32:0/2520227541 >> v2:172.21.15.32:3300/0 conn(0x7fa08c06cac0 0x7fa08c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fd115452700 1 -- 172.21.15.32:0/3429400859 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011830 con 0x7fd110070fd0 2023-11-24T07:58:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fd115452700 1 -- 172.21.15.32:0/3429400859 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100011990 con 0x7fd110070fd0 2023-11-24T07:58:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fd115452700 1 -- 172.21.15.32:0/3429400859 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011d20 con 0x7fd110070fd0 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 0x7f5f20215700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f25cc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f200b4090 0x7f5f20204910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f25cc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f200b4090 0x7f5f20204910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56676/0 (socket says 172.21.15.32:56676) 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 0x7f5f20215700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51924/0 (socket says 172.21.15.32:51924) 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f27f27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f200aee20 0x7f5f202077e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f27f27700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f202152e0 con 0x7f5f200b4090 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f27f27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20204e50 con 0x7f5f200aee20 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20214f90 con 0x7f5f20169190 2023-11-24T07:58:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f25cc3700 1 -- 172.21.15.32:0/219737046 learned_addr learned my addr 172.21.15.32:0/219737046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f467affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 0x7f467c200000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f467a7fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c0b3b50 0x7f467c200540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f467affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 0x7f467c200000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56678/0 (socket says 172.21.15.32:56678) 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f467affd700 1 -- 172.21.15.32:0/2520041662 learned_addr learned my addr 172.21.15.32:0/2520041662 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f467b7fe700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f467c164a90 0x7f467c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038235082 0 0) 0x7fa08c06cf40 con 0x7fa08c08b290 2023-11-24T07:58:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0780035b0 con 0x7fa08c08b290 2023-11-24T07:58:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 -- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f200aee20 msgr2=0x7f5f202077e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f200aee20 0x7f5f202077e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 -- 172.21.15.32:0/219737046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f200b4090 msgr2=0x7f5f20204910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f200b4090 0x7f5f20204910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f5f254c2700 1 -- 172.21.15.32:0/219737046 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009b30 con 0x7f5f20169190 2023-11-24T07:58:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2154820767 0 0) 0x7fa0780035b0 con 0x7fa08c08b290 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 >> v2:172.21.15.32:3300/0 conn(0x7fa08c06cac0 msgr2=0x7fa08c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 --2- 172.21.15.32:0/2520227541 >> v2:172.21.15.32:3300/0 conn(0x7fa08c06cac0 0x7fa08c0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 >> v1:172.21.15.32:6789/0 conn(0x7fa08c16bf80 legacy=0x7fa08c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c0b4680 con 0x7fa08c08b290 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fa088001670 con 0x7fa08c08b290 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 >> v1:172.21.15.123:6789/0 conn(0x7fa08c08b290 legacy=0x7fa08c08b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa078003a00 0x7fa078003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa078004850 0x7fa078004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa078010020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0b43a0 con 0x7fa078003a00 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa078010590 con 0x7fa078004850 2023-11-24T07:58:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07800bc10 con 0x7fa078005210 2023-11-24T07:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f467affd700 1 -- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c0b3b50 msgr2=0x7f467c200540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f467affd700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c0b3b50 0x7f467c200540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f467affd700 1 -- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f467c164a90 msgr2=0x7f467c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f467affd700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f467c164a90 0x7f467c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f467affd700 1 -- 172.21.15.32:0/2520041662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0094a0 con 0x7f467c06d4d0 2023-11-24T07:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f5f254c2700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 0x7f5f20215700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0062d0 tx=0x7f5f1c0065e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa091bd3700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa078010020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f9f537fe700 1 -- 172.21.15.32:0/3704127205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48004230 con 0x7f9f5415ff10 2023-11-24T07:58:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f9f537fe700 1 -- 172.21.15.32:0/3704127205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48004390 con 0x7f9f5415ff10 2023-11-24T07:58:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7f9f537fe700 1 -- 172.21.15.32:0/3704127205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480036a0 con 0x7f9f5415ff10 2023-11-24T07:58:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f9f5b022700 1 -- 172.21.15.32:0/3704127205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 msgr2=0x7f9f540c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f9f5b022700 1 --2- 172.21.15.32:0/3704127205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f540c4cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009840 tx=0x7f9f48009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c006cf0 con 0x7f5f20169190 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c006e50 con 0x7f5f20169190 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c010920 con 0x7f5f20169190 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200cd440 con 0x7f5f20169190 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa0913d2700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa078003a00 0x7fa078003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa091bd3700 1 -- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa078004850 msgr2=0x7fa078004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T07:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa091bd3700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa078004850 0x7fa078004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7fa091bd3700 1 -- 172.21.15.32:0/2520227541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa078003a00 msgr2=0x7fa078003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f467affd700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 0x7f467c200000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f467000c780 tx=0x7f467000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa091bd3700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa078003a00 0x7fa078003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa091bd3700 1 -- 172.21.15.32:0/2520227541 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c0b4680 con 0x7fa078005210 2023-11-24T07:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa091bd3700 1 -- 172.21.15.32:0/2520227541 reap_dead start 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f9f5b022700 1 -- 172.21.15.32:0/3704127205 shutdown_connections 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f9f5b022700 1 --2- 172.21.15.32:0/3704127205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f540c4cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f9f5b022700 1 -- 172.21.15.32:0/3704127205 >> 172.21.15.32:0/3704127205 conn(0x7f9f540731a0 msgr2=0x7f9f5409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f72961e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72900b3b50 0x7f72900b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f72961e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f729006d4d0 0x7f729006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f72961e3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7290163a80 0x7f7290163f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f72961e3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729006ced0 con 0x7f72900b3b50 2023-11-24T07:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f72961e3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729006da70 con 0x7f7290163a80 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f72961e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b4470 con 0x7f729006d4d0 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f728f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72900b3b50 0x7f72900b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f728f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72900b3b50 0x7f72900b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56692/0 (socket says 172.21.15.32:56692) 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f728f7fe700 1 -- 172.21.15.32:0/3503834734 learned_addr learned my addr 172.21.15.32:0/3503834734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f728ffff700 1 --2- 172.21.15.32:0/3503834734 >> v2:172.21.15.123:3300/0 conn(0x7f7290163a80 0x7f7290163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f728f7fe700 1 -- 172.21.15.32:0/3503834734 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f729006d4d0 msgr2=0x7f729006eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f728f7fe700 1 --2- 172.21.15.32:0/3503834734 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f729006d4d0 0x7f729006eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f728f7fe700 1 -- 172.21.15.32:0/3503834734 >> v2:172.21.15.123:3300/0 conn(0x7f7290163a80 msgr2=0x7f7290163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f728f7fe700 1 --2- 172.21.15.32:0/3503834734 >> v2:172.21.15.123:3300/0 conn(0x7f7290163a80 0x7f7290163f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f728f7fe700 1 -- 172.21.15.32:0/3503834734 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72901634e0 con 0x7f72900b3b50 2023-11-24T07:58:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f728f7fe700 1 --2- 172.21.15.32:0/3503834734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72900b3b50 0x7f72900b3f30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72900706f0 tx=0x7f728800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dee0b4f76b505e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 -- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd110070fd0 msgr2=0x7fd110165e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd110070fd0 0x7fd110165e90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098f0 tx=0x7fd100009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 -- 172.21.15.32:0/3429400859 shutdown_connections 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1101663d0 0x7fd1100b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3429400859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd110070fd0 0x7fd110165e90 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098f0 tx=0x7fd100009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 -- 172.21.15.32:0/3429400859 >> 172.21.15.32:0/3429400859 conn(0x7fd1100b1740 msgr2=0x7fd1100b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 -- 172.21.15.32:0/3429400859 shutdown_connections 2023-11-24T07:58:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa0913d2700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa078003a00 0x7fa078003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa091bd3700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa078010020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa08400c190 tx=0x7fa078005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a9e83a7ef3694f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400ccd0 con 0x7fa078005210 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08400ce30 con 0x7fa078005210 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fa083fff700 1 -- 172.21.15.32:0/2520227541 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084003c40 con 0x7fa078005210 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fab1ac50700 1 -- 172.21.15.32:0/1924308822 shutdown_connections 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.974+0000 7fab1ac50700 1 -- 172.21.15.32:0/1924308822 wait complete. 2023-11-24T07:58:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fab1ac50700 1 Processor -- start 2023-11-24T07:58:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.975+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200cda00 con 0x7f5f20169190 2023-11-24T07:58:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670004c80 con 0x7f467c06d4d0 2023-11-24T07:58:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4670004e00 con 0x7f467c06d4d0 2023-11-24T07:58:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670007310 con 0x7f467c06d4d0 2023-11-24T07:58:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fd1186b8700 1 -- 172.21.15.32:0/3429400859 wait complete. 2023-11-24T07:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fa093636700 1 -- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 msgr2=0x7fa078010020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fa093636700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa078010020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa08400c190 tx=0x7fa078005880 comp rx=0 tx=0).stop 2023-11-24T07:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.976+0000 7fab1ac50700 1 -- start start 2023-11-24T07:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c071430 con 0x7f467c06d4d0 2023-11-24T07:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c0b0fb0 con 0x7f467c06d4d0 2023-11-24T07:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fa093636700 1 -- 172.21.15.32:0/2520227541 shutdown_connections 2023-11-24T07:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fa093636700 1 --2- 172.21.15.32:0/2520227541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa078010020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fa093636700 1 -- 172.21.15.32:0/2520227541 >> 172.21.15.32:0/2520227541 conn(0x7fa08c0cfa30 msgr2=0x7fa08c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fab1ac50700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc0050d0 0x7fab141552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fab1ac50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 0x7fab140b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fab1ac50700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab140b0b40 0x7fab141579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fab1ac50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b0fc0 con 0x7fab141563b0 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fab1ac50700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14156920 con 0x7fab140b0b40 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.977+0000 7fab1ac50700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab14156c00 con 0x7faafc0050d0 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab191ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab140b0b40 0x7fab141579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 0x7fab140b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 0x7fab140b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56702/0 (socket says 172.21.15.32:56702) 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 -- 172.21.15.32:0/4115486877 learned_addr learned my addr 172.21.15.32:0/4115486877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fd1186b8700 1 Processor -- start 2023-11-24T07:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 -- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc0050d0 msgr2=0x7fab141552d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc0050d0 0x7fab141552d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 -- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab140b0b40 msgr2=0x7fab141579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab140b0b40 0x7fab141579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 -- 172.21.15.32:0/4115486877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14156ee0 con 0x7fab141563b0 2023-11-24T07:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fab13fff700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 0x7fab140b0600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab0800c570 tx=0x7fab0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7f9f5b022700 1 -- 172.21.15.32:0/3704127205 shutdown_connections 2023-11-24T07:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7f728e7fc700 1 -- 172.21.15.32:0/3503834734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800e070 con 0x7f72900b3b50 2023-11-24T07:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7f728e7fc700 1 -- 172.21.15.32:0/3503834734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728800b9c0 con 0x7f72900b3b50 2023-11-24T07:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7f728e7fc700 1 -- 172.21.15.32:0/3503834734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800bd70 con 0x7f72900b3b50 2023-11-24T07:58:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fa093636700 1 -- 172.21.15.32:0/2520227541 shutdown_connections 2023-11-24T07:58:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c15aed0 con 0x7f467c06d4d0 2023-11-24T07:58:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.978+0000 7fd1186b8700 1 -- start start 2023-11-24T07:58:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fd1186b8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1100cf440 0x7fd11013e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fd1186b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101663d0 0x7fd1100b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fd1186b8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1100b0730 0x7fd110140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fd1186b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100b0bc0 con 0x7fd1101663d0 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fd1186b8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11015f3a0 con 0x7fd1100b0730 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fd1186b8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11015f680 con 0x7fd1100cf440 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.979+0000 7fa093636700 1 -- 172.21.15.32:0/2520227541 wait complete. 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08004220 con 0x7fab141563b0 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08004380 con 0x7fab141563b0 2023-11-24T07:58:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080036a0 con 0x7fab141563b0 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab141571c0 con 0x7fab141563b0 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab14155810 con 0x7fab141563b0 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fd116c55700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1100b0730 0x7fd110140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fd116454700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1100cf440 0x7fd11013e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fd116454700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1100cf440 0x7fd11013e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51956/0 (socket says 172.21.15.32:51956) 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fd116454700 1 -- 172.21.15.32:0/3796558788 learned_addr learned my addr 172.21.15.32:0/3796558788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7fd115c53700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101663d0 0x7fd1100b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f080052e0 con 0x7f5f20169190 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 -- 172.21.15.32:0/3503834734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72900b3b50 msgr2=0x7f72900b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 --2- 172.21.15.32:0/3503834734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72900b3b50 0x7f72900b3f30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72900706f0 tx=0x7f728800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 -- 172.21.15.32:0/3503834734 shutdown_connections 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 --2- 172.21.15.32:0/3503834734 >> v2:172.21.15.123:3300/0 conn(0x7f7290163a80 0x7f7290163f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 --2- 172.21.15.32:0/3503834734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72900b3b50 0x7f72900b3f30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 --2- 172.21.15.32:0/3503834734 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f729006d4d0 0x7f729006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 -- 172.21.15.32:0/3503834734 >> 172.21.15.32:0/3503834734 conn(0x7f72900cfa30 msgr2=0x7f72900cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 -- 172.21.15.32:0/3503834734 shutdown_connections 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f72961e3700 1 -- 172.21.15.32:0/3503834734 wait complete. 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fd115c53700 1 -- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1100cf440 msgr2=0x7fd11013e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fd115c53700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1100cf440 0x7fd11013e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fd115c53700 1 -- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1100b0730 msgr2=0x7fd110140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fd115c53700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1100b0730 0x7fd110140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fd115c53700 1 -- 172.21.15.32:0/3796558788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd100009550 con 0x7fd1101663d0 2023-11-24T07:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4670007470 con 0x7f467c06d4d0 2023-11-24T07:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f4663fff700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f466806afa0 0x7f466806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.980+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4670088ba0 con 0x7f467c06d4d0 2023-11-24T07:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7f467a7fc700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f466806afa0 0x7f466806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fd115c53700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101663d0 0x7fd1100b01f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd100016000 tx=0x7fd100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7fa093636700 1 Processor -- start 2023-11-24T07:58:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7f467a7fc700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f466806afa0 0x7f466806d460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4674007fd0 tx=0x7f4674009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.982+0000 7f72961e3700 1 Processor -- start 2023-11-24T07:58:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.982+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab140751e0 con 0x7fab141563b0 2023-11-24T07:58:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.982+0000 7f72961e3700 1 -- start start 2023-11-24T07:58:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.981+0000 7f9f5b022700 1 -- 172.21.15.32:0/3704127205 wait complete. 2023-11-24T07:58:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.982+0000 7f9f5b022700 1 Processor -- start 2023-11-24T07:58:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.982+0000 7f9f5b022700 1 -- start start 2023-11-24T07:58:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.982+0000 7fa093636700 1 -- start start 2023-11-24T07:58:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa093636700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa08c161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa093636700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c160d20 0x7fa08c15f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa093636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 0x7fa08c160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa093636700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c161510 con 0x7fa08c15fcc0 2023-11-24T07:58:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa093636700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c16d390 con 0x7fa08c160d20 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa093636700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c166ca0 con 0x7fa078005210 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f72961e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 0x7f72901590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f72961e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72900b3b50 0x7f729015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f72961e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7290163a80 0x7f729015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f72961e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290158c60 con 0x7f7290163a80 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f72961e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290158050 con 0x7f729006d4d0 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f72961e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7290158330 con 0x7f72900b3b50 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f728ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7290163a80 0x7f729015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f728f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 0x7f72901590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f728ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7290163a80 0x7f729015cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56716/0 (socket says 172.21.15.32:56716) 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f728f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 0x7f72901590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40862/0 (socket says 172.21.15.32:40862) 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f728ffff700 1 -- 172.21.15.32:0/2552876895 learned_addr learned my addr 172.21.15.32:0/2552876895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f728effd700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72900b3b50 0x7f729015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa0913d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa08c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa091bd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 0x7fa08c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa091bd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 0x7fa08c160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56720/0 (socket says 172.21.15.32:56720) 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa0913d2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa08c161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:51970/0 (socket says 172.21.15.32:51970) 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7fa091bd3700 1 -- 172.21.15.32:0/1836711296 learned_addr learned my addr 172.21.15.32:0/1836711296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa090bd1700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c160d20 0x7fa08c15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.983+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f1c010a80 con 0x7f5f20169190 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f5f167fc700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5f0406afa0 0x7f5f0406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f728f7fe700 1 -- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72900b3b50 msgr2=0x7f729015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f728f7fe700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72900b3b50 0x7f729015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f728f7fe700 1 -- 172.21.15.32:0/2552876895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7290163a80 msgr2=0x7f729015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f728f7fe700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7290163a80 0x7f729015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f728f7fe700 1 -- 172.21.15.32:0/2552876895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288009b30 con 0x7f729006d4d0 2023-11-24T07:58:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f1c089730 con 0x7f5f20169190 2023-11-24T07:58:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f5b022700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540b3370 0x7f9f5414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f5b022700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f540cc330 0x7f9f5414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f5b022700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f5414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f5b022700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540cb6d0 con 0x7f9f5415ff10 2023-11-24T07:58:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f5b022700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5415c7a0 con 0x7f9f540b3370 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f5b022700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540cb970 con 0x7f9f540cc330 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f58dbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540b3370 0x7f9f5414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa091bd3700 1 -- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 msgr2=0x7fa08c161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa091bd3700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa08c161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa091bd3700 1 -- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c160d20 msgr2=0x7fa08c15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa091bd3700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c160d20 0x7fa08c15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f728f7fe700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 0x7f72901590f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7288015000 tx=0x7f7288004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa091bd3700 1 -- 172.21.15.32:0/1836711296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c166f80 con 0x7fa08c15fcc0 2023-11-24T07:58:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7fa090bd1700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c160d20 0x7fa08c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f58dbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540b3370 0x7f9f5414a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:40876/0 (socket says 172.21.15.32:40876) 2023-11-24T07:58:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f58dbe700 1 -- 172.21.15.32:0/350021814 learned_addr learned my addr 172.21.15.32:0/350021814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f5f25cc3700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5f0406afa0 0x7f5f0406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.984+0000 7f9f595bf700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f5414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288004250 con 0x7f729006d4d0 2023-11-24T07:58:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72880043d0 con 0x7f729006d4d0 2023-11-24T07:58:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7290158610 con 0x7f729006d4d0 2023-11-24T07:58:45.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288004780 con 0x7f729006d4d0 2023-11-24T07:58:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7fa091bd3700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 0x7fa08c160140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa08c04f460 tx=0x7fa0840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729015a830 con 0x7f729006d4d0 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f53fff700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f540cc330 0x7f9f5414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f595bf700 1 -- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f540cc330 msgr2=0x7f9f5414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f595bf700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f540cc330 0x7f9f5414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f595bf700 1 -- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540b3370 msgr2=0x7f9f5414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f595bf700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540b3370 0x7f9f5414a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f595bf700 1 -- 172.21.15.32:0/350021814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480094a0 con 0x7f9f5415ff10 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f53fff700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f540cc330 0x7f9f5414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f9f595bf700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f5414f0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00cae0 tx=0x7f9f4c00cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.985+0000 7f5f25cc3700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5f0406afa0 0x7f5f0406d460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b370 tx=0x7f5f18009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.986+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7290158f40 con 0x7f729006d4d0 2023-11-24T07:58:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.986+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f728801c440 con 0x7f729006d4d0 2023-11-24T07:58:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.987+0000 7f7277fff700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f727c06af20 0x7f727c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.987+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f728808a790 con 0x7f729006d4d0 2023-11-24T07:58:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.987+0000 7f728effd700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f727c06af20 0x7f727c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.987+0000 7f728effd700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f727c06af20 0x7f727c06d3e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7290153b90 tx=0x7f728401a0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.988+0000 7f1229628700 1 Processor -- start 2023-11-24T07:58:45.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.988+0000 7f1229628700 1 -- start start 2023-11-24T07:58:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.988+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4670053c30 con 0x7f467c06d4d0 2023-11-24T07:58:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.988+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100004450 con 0x7fd1101663d0 2023-11-24T07:58:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.988+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10001f070 con 0x7fd1101663d0 2023-11-24T07:58:45.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.988+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1000038c0 con 0x7fd1101663d0 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f1229628700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f122406f460 0x7f12240b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f1229628700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122406bf50 con 0x7f12240b4ef0 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f1229628700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122415b030 con 0x7f122415ac50 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f1229628700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12240b5360 con 0x7f122406f460 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f1222ffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f122415ac50 0x7f122406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45200/0 (socket says 172.21.15.32:45200) 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f1222ffd700 1 -- 172.21.15.32:0/1697296297 learned_addr learned my addr 172.21.15.32:0/1697296297 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f12227fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f122406f460 0x7f12240b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084005270 con 0x7fa08c15fcc0 2023-11-24T07:58:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f12227fc700 1 -- 172.21.15.32:0/1697296297 >> v1:172.21.15.123:6789/0 conn(0x7f122415ac50 legacy=0x7f122406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.989+0000 7f12227fc700 1 -- 172.21.15.32:0/1697296297 >> v1:172.21.15.32:6789/0 conn(0x7f12240b4ef0 legacy=0x7f12240cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7f12227fc700 1 -- 172.21.15.32:0/1697296297 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122406e810 con 0x7f122406f460 2023-11-24T07:58:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0840053d0 con 0x7fa08c15fcc0 2023-11-24T07:58:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c167260 con 0x7fa08c15fcc0 2023-11-24T07:58:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab08003820 con 0x7fab141563b0 2023-11-24T07:58:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c003c00 con 0x7f9f5415ff10 2023-11-24T07:58:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c003d60 con 0x7f9f5415ff10 2023-11-24T07:58:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f540cbc50 con 0x7f9f5415ff10 2023-11-24T07:58:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c16d820 con 0x7fa08c15fcc0 2023-11-24T07:58:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7fab117fa700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006af20 0x7fab0006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab08089480 con 0x7fab141563b0 2023-11-24T07:58:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5414ac60 con 0x7f9f5415ff10 2023-11-24T07:58:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.990+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00aae0 con 0x7f9f5415ff10 2023-11-24T07:58:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.991+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084005780 con 0x7fa08c15fcc0 2023-11-24T07:58:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.991+0000 7f12227fc700 1 --2- 172.21.15.32:0/1697296297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240b49b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fa802493ff8786 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.991+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f340052e0 con 0x7f9f5415ff10 2023-11-24T07:58:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.991+0000 7fab189ec700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006af20 0x7fab0006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.991+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f1c054850 con 0x7f5f20169190 2023-11-24T07:58:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.992+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c0052e0 con 0x7fa08c15fcc0 2023-11-24T07:58:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.992+0000 7fab189ec700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006af20 0x7fab0006d3e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab040069a0 tx=0x7fab04000f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.994+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11015f960 con 0x7fd1101663d0 2023-11-24T07:58:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.994+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11013e7e0 con 0x7fd1101663d0 2023-11-24T07:58:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.994+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72880558d0 con 0x7f729006d4d0 2023-11-24T07:58:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.995+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab08054610 con 0x7fab141563b0 2023-11-24T07:58:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.996+0000 7f1221ffb700 1 -- 172.21.15.32:0/1697296297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011830 con 0x7f122406f460 2023-11-24T07:58:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.996+0000 7f1221ffb700 1 -- 172.21.15.32:0/1697296297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218011990 con 0x7f122406f460 2023-11-24T07:58:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.996+0000 7f1221ffb700 1 -- 172.21.15.32:0/1697296297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218011d20 con 0x7f122406f460 2023-11-24T07:58:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.997+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f4c00ac40 con 0x7f9f5415ff10 2023-11-24T07:58:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.998+0000 7f9f517fa700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9f4006aed0 0x7f9f4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.998+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f4c089ff0 con 0x7f9f5415ff10 2023-11-24T07:58:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.998+0000 7f9f58dbe700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9f4006aed0 0x7f9f4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.999+0000 7f9f58dbe700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9f4006aed0 0x7f9f4006d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009b90 tx=0x7f9f48008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.998+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa08401c420 con 0x7fa08c15fcc0 2023-11-24T07:58:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.999+0000 7fa081ffb700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa07006af50 0x7fa07006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.999+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa08408af50 con 0x7fa08c15fcc0 2023-11-24T07:58:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.999+0000 7fa0913d2700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa07006af50 0x7fa07006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.999+0000 7f1229628700 1 -- 172.21.15.32:0/1697296297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 msgr2=0x7f12240b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:44.999+0000 7f1229628700 1 --2- 172.21.15.32:0/1697296297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240b49b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.000+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11015fe80 con 0x7fd1101663d0 2023-11-24T07:58:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.000+0000 7fa0913d2700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa07006af50 0x7fa07006d410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0505a0 tx=0x7fa088009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.000+0000 7f1229628700 1 -- 172.21.15.32:0/1697296297 shutdown_connections 2023-11-24T07:58:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.000+0000 7f1229628700 1 --2- 172.21.15.32:0/1697296297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240b49b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.000+0000 7f1229628700 1 -- 172.21.15.32:0/1697296297 >> 172.21.15.32:0/1697296297 conn(0x7f12240b3130 msgr2=0x7f12240d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.000+0000 7f1229628700 1 -- 172.21.15.32:0/1697296297 shutdown_connections 2023-11-24T07:58:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.001+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd100003a20 con 0x7fd1101663d0 2023-11-24T07:58:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.001+0000 7fd106ffd700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd0f406aed0 0x7fd0f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.001+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd100021070 con 0x7fd1101663d0 2023-11-24T07:58:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.001+0000 7f1229628700 1 -- 172.21.15.32:0/1697296297 wait complete. 2023-11-24T07:58:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.003+0000 7f1229628700 1 Processor -- start 2023-11-24T07:58:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.003+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa084056210 con 0x7fa08c15fcc0 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.003+0000 7f1229628700 1 -- start start 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1229628700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1229628700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12240b4ef0 0x7f1224146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1229628700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122415ac50 0x7f1224158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1229628700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1224157ad0 con 0x7f122406f460 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1229628700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241577f0 con 0x7f122415ac50 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1229628700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122415b390 con 0x7f12240b4ef0 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1222ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1222ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56746/0 (socket says 172.21.15.32:56746) 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f1222ffd700 1 -- 172.21.15.32:0/2240820303 learned_addr learned my addr 172.21.15.32:0/2240820303 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f12237fe700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122415ac50 0x7f1224158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.004+0000 7f12227fc700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12240b4ef0 0x7f1224146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.005+0000 7f1222ffd700 1 -- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12240b4ef0 msgr2=0x7f1224146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.005+0000 7f1222ffd700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12240b4ef0 0x7f1224146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.005+0000 7f1222ffd700 1 -- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122415ac50 msgr2=0x7f1224158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.005+0000 7f1222ffd700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122415ac50 0x7f1224158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.005+0000 7f1222ffd700 1 -- 172.21.15.32:0/2240820303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180094a0 con 0x7f122406f460 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.005+0000 7f1222ffd700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240d01f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f121400c480 tx=0x7f121400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.008+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f4c055080 con 0x7f9f5415ff10 2023-11-24T07:58:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.008+0000 7fd116454700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd0f406aed0 0x7fd0f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.009+0000 7fd116454700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd0f406aed0 0x7fd0f406d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f60 tx=0x7fd10c007f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.013+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd11015fe80 con 0x7fd1101663d0 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.018+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214005d70 con 0x7f122406f460 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.018+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121400ce90 con 0x7f122406f460 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.018+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140072d0 con 0x7f122406f460 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.018+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122415b670 con 0x7f122406f460 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.018+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241517c0 con 0x7f122406f460 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.021+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1214007430 con 0x7f122406f460 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.021+0000 7f120b7fe700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f121006afa0 0x7f121006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.022+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1214088810 con 0x7f122406f460 2023-11-24T07:58:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.022+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12040052e0 con 0x7f122406f460 2023-11-24T07:58:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.027+0000 7f12227fc700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f121006afa0 0x7f121006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.032+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1214053950 con 0x7f122406f460 2023-11-24T07:58:45.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.065+0000 7f12227fc700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f121006afa0 0x7f121006d460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12180090b0 tx=0x7f1218008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.394+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5f08005590 con 0x7f5f20169190 2023-11-24T07:58:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.395+0000 7f5f167fc700 1 -- 172.21.15.32:0/219737046 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f1c057e70 con 0x7f5f20169190 2023-11-24T07:58:45.399 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.398+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5f0406afa0 msgr2=0x7f5f0406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.398+0000 7f5f27f27700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5f0406afa0 0x7f5f0406d460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b370 tx=0x7f5f18009000 comp rx=0 tx=0).stop 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.398+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 msgr2=0x7f5f20215700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.398+0000 7f5f27f27700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 0x7f5f20215700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0062d0 tx=0x7f5f1c0065e0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 shutdown_connections 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5f0406afa0 0x7f5f0406d460 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b370 tx=0x7f5f18009000 comp rx=0 tx=0).stop 2023-11-24T07:58:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f200aee20 0x7f5f202077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f200b4090 0x7f5f20204910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 --2- 172.21.15.32:0/219737046 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5f20169190 0x7f5f20215700 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 >> 172.21.15.32:0/219737046 conn(0x7f5f200cfa30 msgr2=0x7f5f20071620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 shutdown_connections 2023-11-24T07:58:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.399+0000 7f5f27f27700 1 -- 172.21.15.32:0/219737046 wait complete. 2023-11-24T07:58:45.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.406+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f729015a9c0 con 0x7f729006d4d0 2023-11-24T07:58:45.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.407+0000 7f7277fff700 1 -- 172.21.15.32:0/2552876895 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f728802a020 con 0x7f729006d4d0 2023-11-24T07:58:45.411 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-11-24T07:58:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.412+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f467c0751e0 con 0x7f467c06d4d0 2023-11-24T07:58:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.413+0000 7f4663fff700 1 -- 172.21.15.32:0/2520041662 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4670018070 con 0x7f467c06d4d0 2023-11-24T07:58:45.416 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f727c06af20 msgr2=0x7f727c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f727c06af20 0x7f727c06d3e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7290153b90 tx=0x7f728401a0f0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 msgr2=0x7f72901590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 0x7f72901590f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7288015000 tx=0x7f7288004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 shutdown_connections 2023-11-24T07:58:45.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f727c06af20 0x7f727c06d3e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7290163a80 0x7f729015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729006d4d0 0x7f72901590f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 --2- 172.21.15.32:0/2552876895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72900b3b50 0x7f729015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.415+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 >> 172.21.15.32:0/2552876895 conn(0x7f72900cfa30 msgr2=0x7f7290070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.416+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 shutdown_connections 2023-11-24T07:58:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.416+0000 7f72961e3700 1 -- 172.21.15.32:0/2552876895 wait complete. 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f466806afa0 msgr2=0x7f466806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f466806afa0 0x7f466806d460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4674007fd0 tx=0x7f4674009040 comp rx=0 tx=0).stop 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 msgr2=0x7f467c200000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 0x7f467c200000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f467000c780 tx=0x7f467000cb40 comp rx=0 tx=0).stop 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 shutdown_connections 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f466806afa0 0x7f466806d460 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4674007fd0 tx=0x7f4674009040 comp rx=0 tx=0).stop 2023-11-24T07:58:45.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f467c164a90 0x7f467c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f467c06d4d0 0x7f467c200000 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f467000c780 tx=0x7f467000cb40 comp rx=0 tx=0).stop 2023-11-24T07:58:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 --2- 172.21.15.32:0/2520041662 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f467c0b3b50 0x7f467c200540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.417+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 >> 172.21.15.32:0/2520041662 conn(0x7f467c0cfa30 msgr2=0x7f467c0b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-11-24T07:58:45.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.419+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd11015fe80 con 0x7fd1101663d0 2023-11-24T07:58:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.419+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 shutdown_connections 2023-11-24T07:58:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.419+0000 7f46819d7700 1 -- 172.21.15.32:0/2520041662 wait complete. 2023-11-24T07:58:45.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.419+0000 7fd106ffd700 1 -- 172.21.15.32:0/3796558788 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd11015fe80 con 0x7fd1101663d0 2023-11-24T07:58:45.422 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-11-24T07:58:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.420+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fab140b35c0 con 0x7fab141563b0 2023-11-24T07:58:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.420+0000 7fab117fa700 1 -- 172.21.15.32:0/4115486877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab0801e020 con 0x7fab141563b0 2023-11-24T07:58:45.423 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd0f406aed0 msgr2=0x7fd0f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd0f406aed0 0x7fd0f406d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f60 tx=0x7fd10c007f90 comp rx=0 tx=0).stop 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101663d0 msgr2=0x7fd1100b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101663d0 0x7fd1100b01f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd100016000 tx=0x7fd100004750 comp rx=0 tx=0).stop 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 shutdown_connections 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd0f406aed0 0x7fd0f406d390 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f60 tx=0x7fd10c007f90 comp rx=0 tx=0).stop 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1100b0730 0x7fd110140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1100cf440 0x7fd11013e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 --2- 172.21.15.32:0/3796558788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1101663d0 0x7fd1100b01f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 >> 172.21.15.32:0/3796558788 conn(0x7fd1100b1740 msgr2=0x7fd11015b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.425 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-11-24T07:58:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.424+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006af20 msgr2=0x7fab0006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.424+0000 7fab1ac50700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006af20 0x7fab0006d3e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab040069a0 tx=0x7fab04000f50 comp rx=0 tx=0).stop 2023-11-24T07:58:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.424+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 msgr2=0x7fab140b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.424+0000 7fab1ac50700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 0x7fab140b0600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab0800c570 tx=0x7fab0800c880 comp rx=0 tx=0).stop 2023-11-24T07:58:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 shutdown_connections 2023-11-24T07:58:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 shutdown_connections 2023-11-24T07:58:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fab1ac50700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fab0006af20 0x7fab0006d3e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fab1ac50700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab140b0b40 0x7fab141579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fab1ac50700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faafc0050d0 0x7fab141552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fab1ac50700 1 --2- 172.21.15.32:0/4115486877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab141563b0 0x7fab140b0600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 >> 172.21.15.32:0/4115486877 conn(0x7fab140cfa30 msgr2=0x7fab140b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7fd1186b8700 1 -- 172.21.15.32:0/3796558788 wait complete. 2023-11-24T07:58:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9f34005860 con 0x7f9f5415ff10 2023-11-24T07:58:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.422+0000 7f9f517fa700 1 -- 172.21.15.32:0/350021814 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f4c001ba0 con 0x7f9f5415ff10 2023-11-24T07:58:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9f4006aed0 msgr2=0x7f9f4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7f9f5b022700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9f4006aed0 0x7f9f4006d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009b90 tx=0x7f9f48008030 comp rx=0 tx=0).stop 2023-11-24T07:58:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 msgr2=0x7f9f5414f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.425+0000 7f9f5b022700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f5414f0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00cae0 tx=0x7f9f4c00cdf0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.426+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 shutdown_connections 2023-11-24T07:58:45.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.426+0000 7fab1ac50700 1 -- 172.21.15.32:0/4115486877 wait complete. 2023-11-24T07:58:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa06c005860 con 0x7fa08c15fcc0 2023-11-24T07:58:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 shutdown_connections 2023-11-24T07:58:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9f4006aed0 0x7f9f4006d390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f5415ff10 0x7f9f5414f0f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540b3370 0x7f9f5414a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 --2- 172.21.15.32:0/350021814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f540cc330 0x7f9f5414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 >> 172.21.15.32:0/350021814 conn(0x7f9f540731a0 msgr2=0x7f9f540cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 shutdown_connections 2023-11-24T07:58:45.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.433+0000 7f9f5b022700 1 -- 172.21.15.32:0/350021814 wait complete. 2023-11-24T07:58:45.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.434+0000 7fa081ffb700 1 -- 172.21.15.32:0/1836711296 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa084016090 con 0x7fa08c15fcc0 2023-11-24T07:58:45.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-11-24T07:58:45.443 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:45.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-11-24T07:58:45.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-11-24T07:58:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa07006af50 msgr2=0x7fa07006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa07006af50 0x7fa07006d410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0505a0 tx=0x7fa088009040 comp rx=0 tx=0).stop 2023-11-24T07:58:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 msgr2=0x7fa08c160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 0x7fa08c160140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa08c04f460 tx=0x7fa0840014a0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 shutdown_connections 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa07006af50 0x7fa07006d410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15fcc0 0x7fa08c160140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa078005210 0x7fa08c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 --2- 172.21.15.32:0/1836711296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c160d20 0x7fa08c15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.445+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 >> 172.21.15.32:0/1836711296 conn(0x7fa08c0cfa30 msgr2=0x7fa08c1642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.446+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 shutdown_connections 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.446+0000 7fa093636700 1 -- 172.21.15.32:0/1836711296 wait complete. 2023-11-24T07:58:45.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.448+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1204005590 con 0x7f122406f460 2023-11-24T07:58:45.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.448+0000 7f120b7fe700 1 -- 172.21.15.32:0/2240820303 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1214017020 con 0x7f122406f460 2023-11-24T07:58:45.450 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:45.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-11-24T07:58:45.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.452+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f121006afa0 msgr2=0x7f121006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.452+0000 7f1229628700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f121006afa0 0x7f121006d460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12180090b0 tx=0x7f1218008030 comp rx=0 tx=0).stop 2023-11-24T07:58:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.452+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 msgr2=0x7f12240d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.452+0000 7f1229628700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240d01f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f121400c480 tx=0x7f121400c790 comp rx=0 tx=0).stop 2023-11-24T07:58:45.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 shutdown_connections 2023-11-24T07:58:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f121006afa0 0x7f121006d460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2023-11-24T07:58:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122415ac50 0x7f1224158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f122406f460 0x7f12240d01f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 --2- 172.21.15.32:0/2240820303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12240b4ef0 0x7f1224146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 >> 172.21.15.32:0/2240820303 conn(0x7f12240b3130 msgr2=0x7f12240b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 shutdown_connections 2023-11-24T07:58:45.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:45.453+0000 7f1229628700 1 -- 172.21.15.32:0/2240820303 wait complete. 2023-11-24T07:58:45.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-11-24T07:58:45.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-11-24T07:58:46.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T07:58:46.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T07:58:46.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T07:58:46.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T07:58:46.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T07:58:46.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T07:58:46.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T07:58:46.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T07:58:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.643+0000 7f84bfab2700 1 Processor -- start 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.644+0000 7f84bfab2700 1 -- start start 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.645+0000 7f84bfab2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84b816ae60 0x7f84b806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.645+0000 7f84bfab2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b80b4260 0x7f84b8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.645+0000 7f84bfab2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b808bdb0 con 0x7f84b8072bb0 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.645+0000 7f84bfab2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b806ce30 con 0x7f84b816ae60 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.645+0000 7f84bfab2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b806d110 con 0x7f84b80b4260 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.647+0000 7f84be04f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b80b4260 0x7f84b8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.647+0000 7f84be04f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b80b4260 0x7f84b8070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:52010/0 (socket says 172.21.15.32:52010) 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.647+0000 7f84be04f700 1 -- 172.21.15.32:0/1025885913 learned_addr learned my addr 172.21.15.32:0/1025885913 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.650+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159642833 0 0) 0x7f84b808bdb0 con 0x7f84b8072bb0 2023-11-24T07:58:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.651+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a40035b0 con 0x7f84b8072bb0 2023-11-24T07:58:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.651+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046719020 0 0) 0x7f84a40035b0 con 0x7f84b8072bb0 2023-11-24T07:58:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.651+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b80b4260 msgr2=0x7f84b8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.651+0000 7f84bc84c700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b80b4260 0x7f84b8070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.651+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 >> v2:172.21.15.32:3300/0 conn(0x7f84b816ae60 msgr2=0x7f84b806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.648+0000 7f84bd84e700 1 --2- 172.21.15.32:0/1025885913 >> v2:172.21.15.32:3300/0 conn(0x7f84b816ae60 0x7f84b806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 --2- 172.21.15.32:0/1025885913 >> v2:172.21.15.32:3300/0 conn(0x7f84b816ae60 0x7f84b806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b806d3f0 con 0x7f84b8072bb0 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f84a8001670 con 0x7f84b8072bb0 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 >> v1:172.21.15.32:6789/0 conn(0x7f84b8072bb0 legacy=0x7f84b80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4003a00 0x7f84a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84a4004850 0x7f84a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84a4005210 0x7f84a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b806ce30 con 0x7f84a4005210 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b806d110 con 0x7f84a4003a00 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a400c080 con 0x7f84a4004850 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.652+0000 7f84bd84e700 1 --2- 172.21.15.32:0/1025885913 >> v2:172.21.15.32:3300/0 conn(0x7f84b816ae60 0x7f84b806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.653+0000 7f84bd04d700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84a4004850 0x7f84a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.653+0000 7f84bd84e700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4003a00 0x7f84a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bd84e700 1 -- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84a4004850 msgr2=0x7f84a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bd84e700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84a4004850 0x7f84a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bd84e700 1 -- 172.21.15.32:0/1025885913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84a4005210 msgr2=0x7f84a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bd84e700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84a4005210 0x7f84a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bd84e700 1 -- 172.21.15.32:0/1025885913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b806d3f0 con 0x7f84a4003a00 2023-11-24T07:58:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bd84e700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4003a00 0x7f84a4003e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e6a0 tx=0x7f84a4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c38ab1187ecbc57 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007ab0 con 0x7f84a4003a00 2023-11-24T07:58:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b000a440 con 0x7f84a4003a00 2023-11-24T07:58:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.654+0000 7f84bc84c700 1 -- 172.21.15.32:0/1025885913 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000a7b0 con 0x7f84a4003a00 2023-11-24T07:58:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.655+0000 7f84bfab2700 1 -- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4003a00 msgr2=0x7f84a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.655+0000 7f84bfab2700 1 --2- 172.21.15.32:0/1025885913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4003a00 0x7f84a4003e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e6a0 tx=0x7f84a4005880 comp rx=0 tx=0).stop 2023-11-24T07:58:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.657+0000 7f84be04f700 1 -- 172.21.15.32:0/1025885913 reap_dead start 2023-11-24T07:58:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.657+0000 7f84bfab2700 1 -- 172.21.15.32:0/1025885913 shutdown_connections 2023-11-24T07:58:46.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.657+0000 7f84bfab2700 1 -- 172.21.15.32:0/1025885913 >> 172.21.15.32:0/1025885913 conn(0x7f84b80cfa30 msgr2=0x7f84b80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.657+0000 7f84bfab2700 1 -- 172.21.15.32:0/1025885913 shutdown_connections 2023-11-24T07:58:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.657+0000 7f84bfab2700 1 -- 172.21.15.32:0/1025885913 wait complete. 2023-11-24T07:58:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.659+0000 7f84bfab2700 1 Processor -- start 2023-11-24T07:58:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.660+0000 7f84bfab2700 1 -- start start 2023-11-24T07:58:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bfab2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 0x7f84b81637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bfab2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b80cd1c0 0x7f84b80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bfab2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b815e1c0 0x7f84b815e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bfab2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80ce590 con 0x7f84b815dde0 2023-11-24T07:58:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bfab2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80ceef0 con 0x7f84b80cd1c0 2023-11-24T07:58:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bfab2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8209150 con 0x7f84b815e1c0 2023-11-24T07:58:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84be04f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b815e1c0 0x7f84b815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bd84e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 0x7f84b81637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bd84e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 0x7f84b81637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56768/0 (socket says 172.21.15.32:56768) 2023-11-24T07:58:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.661+0000 7f84bd84e700 1 -- 172.21.15.32:0/58546691 learned_addr learned my addr 172.21.15.32:0/58546691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.662+0000 7f84bd84e700 1 -- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b815e1c0 msgr2=0x7f84b815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.662+0000 7f84bd84e700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b815e1c0 0x7f84b815e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.662+0000 7f84bd84e700 1 -- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b80cd1c0 msgr2=0x7f84b80ce9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.662+0000 7f84bd84e700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b80cd1c0 0x7f84b80ce9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.662+0000 7f84bd84e700 1 -- 172.21.15.32:0/58546691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80cd640 con 0x7f84b815dde0 2023-11-24T07:58:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.662+0000 7f84bd84e700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 0x7f84b81637c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f84b000ec90 tx=0x7f84b0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.667+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0023070 con 0x7f84b815dde0 2023-11-24T07:58:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.667+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b000e020 con 0x7f84b815dde0 2023-11-24T07:58:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.667+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000d450 con 0x7f84b815dde0 2023-11-24T07:58:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.667+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b80cd910 con 0x7f84b815dde0 2023-11-24T07:58:46.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.667+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b80cded0 con 0x7f84b815dde0 2023-11-24T07:58:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.674+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84b001d040 con 0x7f84b815dde0 2023-11-24T07:58:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.674+0000 7f84ae7fc700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f84a406c800 0x7f84a406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.674+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84b008ae40 con 0x7f84b815dde0 2023-11-24T07:58:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.674+0000 7f84bd04d700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f84a406c800 0x7f84a406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.675+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a00052e0 con 0x7f84b815dde0 2023-11-24T07:58:46.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.675+0000 7f84bd04d700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f84a406c800 0x7f84a406ecc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000fb0 tx=0x7f84a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.678+0000 7f8821cc1700 1 Processor -- start 2023-11-24T07:58:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.678+0000 7f8821cc1700 1 -- start start 2023-11-24T07:58:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f8821cc1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f881c15d6a0 0x7f881c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f8821cc1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f881c0b4090 0x7f881c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f8821cc1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881c0720e0 con 0x7f881c071a10 2023-11-24T07:58:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f8821cc1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881c072d60 con 0x7f881c15d6a0 2023-11-24T07:58:46.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f8821cc1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881c06de80 con 0x7f881c0b4090 2023-11-24T07:58:46.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f881bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f881c071a10 0x7f881c06cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58488/0 (socket says 172.21.15.32:58488) 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f881bfff700 1 -- 172.21.15.32:0/3252049706 learned_addr learned my addr 172.21.15.32:0/3252049706 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.680+0000 7f881affd700 1 --2- 172.21.15.32:0/3252049706 >> v2:172.21.15.123:3300/0 conn(0x7f881c0b4090 0x7f881c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.681+0000 7f881b7fe700 1 --2- 172.21.15.32:0/3252049706 >> v2:172.21.15.32:3300/0 conn(0x7f881c15d6a0 0x7f881c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.682+0000 7f881b7fe700 1 -- 172.21.15.32:0/3252049706 >> v2:172.21.15.123:3300/0 conn(0x7f881c0b4090 msgr2=0x7f881c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.682+0000 7f881b7fe700 1 --2- 172.21.15.32:0/3252049706 >> v2:172.21.15.123:3300/0 conn(0x7f881c0b4090 0x7f881c0714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.682+0000 7f881b7fe700 1 -- 172.21.15.32:0/3252049706 >> v1:172.21.15.32:6789/0 conn(0x7f881c071a10 legacy=0x7f881c06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.682+0000 7f881b7fe700 1 -- 172.21.15.32:0/3252049706 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0723c0 con 0x7f881c15d6a0 2023-11-24T07:58:46.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.682+0000 7f881b7fe700 1 --2- 172.21.15.32:0/3252049706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c15d6a0 0x7f881c0b3b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f881c062350 tx=0x7f880c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55a456f9da3ee89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.684+0000 7f238a5b5700 1 Processor -- start 2023-11-24T07:58:46.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.684+0000 7f238a5b5700 1 -- start start 2023-11-24T07:58:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.685+0000 7f881a7fc700 1 -- 172.21.15.32:0/3252049706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c00e070 con 0x7f881c15d6a0 2023-11-24T07:58:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.685+0000 7f881a7fc700 1 -- 172.21.15.32:0/3252049706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880c00bbd0 con 0x7f881c15d6a0 2023-11-24T07:58:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.685+0000 7f881a7fc700 1 -- 172.21.15.32:0/3252049706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c012430 con 0x7f881c15d6a0 2023-11-24T07:58:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.685+0000 7f238a5b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384162750 0x7f23840cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.685+0000 7f238a5b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2384073360 0x7f23840726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f238a5b5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23840cf5c0 con 0x7f23840c3820 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f238a5b5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2384062440 con 0x7f2384073360 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f238a5b5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2384062720 con 0x7f2384162750 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f2388b52700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f23840c3820 0x7f23840ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45248/0 (socket says 172.21.15.32:45248) 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f2388b52700 1 -- 172.21.15.32:0/1914497566 learned_addr learned my addr 172.21.15.32:0/1914497566 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 -- 172.21.15.32:0/3252049706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c15d6a0 msgr2=0x7f881c0b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3252049706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c15d6a0 0x7f881c0b3b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f881c062350 tx=0x7f880c00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f23837fe700 1 --2- 172.21.15.32:0/1914497566 >> v2:172.21.15.32:3300/0 conn(0x7f2384073360 0x7f23840726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f2383fff700 1 --2- 172.21.15.32:0/1914497566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384162750 0x7f23840cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 -- 172.21.15.32:0/3252049706 shutdown_connections 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3252049706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c15d6a0 0x7f881c0b3b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3252049706 >> v2:172.21.15.123:3300/0 conn(0x7f881c0b4090 0x7f881c0714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 -- 172.21.15.32:0/3252049706 >> 172.21.15.32:0/3252049706 conn(0x7f881c0cfa30 msgr2=0x7f881c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84b0056080 con 0x7f84b815dde0 2023-11-24T07:58:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 -- 172.21.15.32:0/3252049706 shutdown_connections 2023-11-24T07:58:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f8821cc1700 1 -- 172.21.15.32:0/3252049706 wait complete. 2023-11-24T07:58:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f23837fe700 1 -- 172.21.15.32:0/1914497566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384162750 msgr2=0x7f23840cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f23837fe700 1 --2- 172.21.15.32:0/1914497566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384162750 0x7f23840cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f23837fe700 1 -- 172.21.15.32:0/1914497566 >> v1:172.21.15.123:6789/0 conn(0x7f23840c3820 legacy=0x7f23840ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.686+0000 7f23837fe700 1 -- 172.21.15.32:0/1914497566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840cff90 con 0x7f2384073360 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.687+0000 7f23837fe700 1 --2- 172.21.15.32:0/1914497566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384073360 0x7f23840726a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2378009840 tx=0x7f2378009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fccd5ac228e82a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.687+0000 7f2382ffd700 1 -- 172.21.15.32:0/1914497566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378011830 con 0x7f2384073360 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.687+0000 7f2382ffd700 1 -- 172.21.15.32:0/1914497566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2378011990 con 0x7f2384073360 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.687+0000 7f2382ffd700 1 -- 172.21.15.32:0/1914497566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378011d20 con 0x7f2384073360 2023-11-24T07:58:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.688+0000 7f8821cc1700 1 Processor -- start 2023-11-24T07:58:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 -- 172.21.15.32:0/1914497566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384073360 msgr2=0x7f23840726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1914497566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384073360 0x7f23840726a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2378009840 tx=0x7f2378009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.688+0000 7f54b34fc700 1 Processor -- start 2023-11-24T07:58:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.688+0000 7f54b34fc700 1 -- start start 2023-11-24T07:58:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 -- 172.21.15.32:0/1914497566 shutdown_connections 2023-11-24T07:58:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1914497566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384162750 0x7f23840cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1914497566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2384073360 0x7f23840726a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 -- 172.21.15.32:0/1914497566 >> 172.21.15.32:0/1914497566 conn(0x7f23840d0230 msgr2=0x7f23840d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f54b34fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f54ac140680 0x7f54ac140c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 -- 172.21.15.32:0/1914497566 shutdown_connections 2023-11-24T07:58:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f238a5b5700 1 -- 172.21.15.32:0/1914497566 wait complete. 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f54b34fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54ac071470 0x7f54ac0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f54b34fc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac06d550 con 0x7f54ac0b3d40 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f54b34fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac0b47c0 con 0x7f54ac071470 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.689+0000 7f54b34fc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac06cc60 con 0x7f54ac140680 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.688+0000 7f8821cc1700 1 -- start start 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f8821cc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881c071a10 0x7f881c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f8821cc1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 0x7f881c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f8821cc1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f881c15d6a0 0x7f881c0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f8821cc1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0ae1f0 con 0x7f881c0b4090 2023-11-24T07:58:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f8821cc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0ad620 con 0x7f881c071a10 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f8821cc1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0ad8c0 con 0x7f881c15d6a0 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1a99700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54ac071470 0x7f54ac0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1298700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f54ac140680 0x7f54ac140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1a99700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54ac071470 0x7f54ac0b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56792/0 (socket says 172.21.15.32:56792) 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1a99700 1 -- 172.21.15.32:0/829414142 learned_addr learned my addr 172.21.15.32:0/829414142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f881affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 0x7f881c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f881affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 0x7f881c15b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56794/0 (socket says 172.21.15.32:56794) 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f881affd700 1 -- 172.21.15.32:0/3976174439 learned_addr learned my addr 172.21.15.32:0/3976174439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f881bfff700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f881c15d6a0 0x7f881c0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1298700 1 -- 172.21.15.32:0/829414142 >> v2:172.21.15.32:3300/0 conn(0x7f54ac071470 msgr2=0x7f54ac0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1298700 1 --2- 172.21.15.32:0/829414142 >> v2:172.21.15.32:3300/0 conn(0x7f54ac071470 0x7f54ac0b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1298700 1 -- 172.21.15.32:0/829414142 >> v1:172.21.15.123:6789/0 conn(0x7f54ac0b3d40 legacy=0x7f54ac070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1298700 1 -- 172.21.15.32:0/829414142 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac06cf00 con 0x7f54ac140680 2023-11-24T07:58:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f54b1298700 1 --2- 172.21.15.32:0/829414142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac140c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009840 tx=0x7f54a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64eec8015d600310 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.690+0000 7f881b7fe700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881c071a10 0x7f881c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f881affd700 1 -- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f881c15d6a0 msgr2=0x7f881c0aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f881affd700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f881c15d6a0 0x7f881c0aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f881affd700 1 -- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881c071a10 msgr2=0x7f881c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f881affd700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881c071a10 0x7f881c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f881affd700 1 -- 172.21.15.32:0/3976174439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c009a70 con 0x7f881c0b4090 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f54a3fff700 1 -- 172.21.15.32:0/829414142 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8011830 con 0x7f54ac140680 2023-11-24T07:58:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f54a3fff700 1 -- 172.21.15.32:0/829414142 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8011990 con 0x7f54ac140680 2023-11-24T07:58:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f54a3fff700 1 -- 172.21.15.32:0/829414142 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8011d20 con 0x7f54ac140680 2023-11-24T07:58:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f238a5b5700 1 Processor -- start 2023-11-24T07:58:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.691+0000 7f881affd700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 0x7f881c15b2d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f881000d8d0 tx=0x7f881000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.692+0000 7f54b34fc700 1 -- 172.21.15.32:0/829414142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 msgr2=0x7f54ac140c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.692+0000 7f54b34fc700 1 --2- 172.21.15.32:0/829414142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac140c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009840 tx=0x7f54a8009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.692+0000 7f238a5b5700 1 -- start start 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f238a5b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 0x7f2384099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f238a5b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23840c3820 0x7f23841ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f238a5b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2384162750 0x7f23840c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f238a5b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841ee090 con 0x7f23840c3820 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f238a5b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841eee20 con 0x7f2384162750 2023-11-24T07:58:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f238a5b5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23841f8800 con 0x7f2384073360 2023-11-24T07:58:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f54b34fc700 1 -- 172.21.15.32:0/829414142 shutdown_connections 2023-11-24T07:58:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f54b34fc700 1 --2- 172.21.15.32:0/829414142 >> v2:172.21.15.32:3300/0 conn(0x7f54ac071470 0x7f54ac0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f54b34fc700 1 --2- 172.21.15.32:0/829414142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac140c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f54b34fc700 1 -- 172.21.15.32:0/829414142 >> 172.21.15.32:0/829414142 conn(0x7f54ac0b1740 msgr2=0x7f54ac0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 0x7f2384099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 0x7f2384099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:52052/0 (socket says 172.21.15.32:52052) 2023-11-24T07:58:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 -- 172.21.15.32:0/1857481139 learned_addr learned my addr 172.21.15.32:0/1857481139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f23837fe700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23840c3820 0x7f23841ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2388b52700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2384162750 0x7f23840c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 -- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2384162750 msgr2=0x7f23840c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2384162750 0x7f23840c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 -- 172.21.15.32:0/1857481139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23840c3820 msgr2=0x7f23841ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23840c3820 0x7f23841ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.693+0000 7f2383fff700 1 -- 172.21.15.32:0/1857481139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780094a0 con 0x7f2384073360 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.694+0000 7f2383fff700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 0x7f2384099a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f237400e870 tx=0x7f237400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.694+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374009590 con 0x7f2384073360 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.694+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374013680 con 0x7f2384073360 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.694+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740139f0 con 0x7f2384073360 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.694+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23841ed410 con 0x7f2384073360 2023-11-24T07:58:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.694+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841ed9d0 con 0x7f2384073360 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.695+0000 7f54b34fc700 1 -- 172.21.15.32:0/829414142 shutdown_connections 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.695+0000 7f54b34fc700 1 -- 172.21.15.32:0/829414142 wait complete. 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.696+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810011870 con 0x7f881c0b4090 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.696+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88100119d0 con 0x7f881c0b4090 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.696+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810011d40 con 0x7f881c0b4090 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.696+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c0adba0 con 0x7f881c0b4090 2023-11-24T07:58:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.696+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c0cea90 con 0x7f881c0b4090 2023-11-24T07:58:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.697+0000 7f54b34fc700 1 Processor -- start 2023-11-24T07:58:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.697+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2374013b50 con 0x7f2384073360 2023-11-24T07:58:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.697+0000 7f2380ff9700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f237006aed0 0x7f237006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.697+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2374089580 con 0x7f2384073360 2023-11-24T07:58:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f5ecde5f700 1 Processor -- start 2023-11-24T07:58:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f5ecde5f700 1 -- start start 2023-11-24T07:58:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f23837fe700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f237006aed0 0x7f237006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f881000f400 con 0x7f881c0b4090 2023-11-24T07:58:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f8803fff700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f880806af20 0x7f880806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.697+0000 7f54b34fc700 1 -- start start 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f54b34fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac071470 0x7f54ac15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f54b34fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 0x7f54ac0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f54b34fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac0aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f54b34fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac169160 con 0x7f54ac0b3d40 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f54b34fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0ce4a0 con 0x7f54ac140680 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.698+0000 7f54b34fc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac0ce780 con 0x7f54ac071470 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 0x7f54ac0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 0x7f54ac0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56810/0 (socket says 172.21.15.32:56810) 2023-11-24T07:58:46.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 -- 172.21.15.32:0/1047577748 learned_addr learned my addr 172.21.15.32:0/1047577748 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b1a99700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac0aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8810089640 con 0x7f881c0b4090 2023-11-24T07:58:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ecde5f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815dfc0 0x7f5ec80d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ecde5f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5ec80d44e0 0x7f5ec80d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ecde5f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80d2d10 con 0x7f5ec80b2fc0 2023-11-24T07:58:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ecde5f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80cefd0 con 0x7f5ec815dfc0 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ecde5f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80cf2b0 con 0x7f5ec80d44e0 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ec77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815dfc0 0x7f5ec80d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ec77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815dfc0 0x7f5ec80d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56814/0 (socket says 172.21.15.32:56814) 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f5ec77fe700 1 -- 172.21.15.32:0/390992044 learned_addr learned my addr 172.21.15.32:0/390992044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f5ec7fff700 1 --2- 172.21.15.32:0/390992044 >> v2:172.21.15.123:3300/0 conn(0x7f5ec80d44e0 0x7f5ec80d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc0052e0 con 0x7f881c0b4090 2023-11-24T07:58:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f881b7fe700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f880806af20 0x7f880806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 -- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac071470 msgr2=0x7f54ac15fcd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac071470 0x7f54ac15fcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 -- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 msgr2=0x7f54ac0aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac0aac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.699+0000 7f54b0a97700 1 -- 172.21.15.32:0/1047577748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80094a0 con 0x7f54ac0b3d40 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f5ec7fff700 1 -- 172.21.15.32:0/390992044 >> v2:172.21.15.32:3300/0 conn(0x7f5ec815dfc0 msgr2=0x7f5ec80d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f5ec7fff700 1 --2- 172.21.15.32:0/390992044 >> v2:172.21.15.32:3300/0 conn(0x7f5ec815dfc0 0x7f5ec80d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f5ec7fff700 1 -- 172.21.15.32:0/390992044 >> v1:172.21.15.123:6789/0 conn(0x7f5ec80b2fc0 legacy=0x7f5ec8161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f5ec7fff700 1 -- 172.21.15.32:0/390992044 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80d2fb0 con 0x7f5ec80d44e0 2023-11-24T07:58:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.700+0000 7f5ec7fff700 1 --2- 172.21.15.32:0/390992044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80d13a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80caf20 tx=0x7f5ec000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecba7f8add30c424 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.701+0000 7f881b7fe700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f880806af20 0x7f880806d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b370 tx=0x7f880c011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.701+0000 7f5ec67fc700 1 -- 172.21.15.32:0/390992044 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000e070 con 0x7f5ec80d44e0 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.701+0000 7f5ec67fc700 1 -- 172.21.15.32:0/390992044 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000bae0 con 0x7f5ec80d44e0 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.701+0000 7f5ec67fc700 1 -- 172.21.15.32:0/390992044 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000be70 con 0x7f5ec80d44e0 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 -- 172.21.15.32:0/390992044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec80d44e0 msgr2=0x7f5ec80d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/390992044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80d13a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80caf20 tx=0x7f5ec000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 -- 172.21.15.32:0/390992044 shutdown_connections 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/390992044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80d13a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/390992044 >> v2:172.21.15.32:3300/0 conn(0x7f5ec815dfc0 0x7f5ec80d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 -- 172.21.15.32:0/390992044 >> 172.21.15.32:0/390992044 conn(0x7f5ec806b410 msgr2=0x7f5ec806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 -- 172.21.15.32:0/390992044 shutdown_connections 2023-11-24T07:58:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.702+0000 7f5ecde5f700 1 -- 172.21.15.32:0/390992044 wait complete. 2023-11-24T07:58:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.701+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23640052e0 con 0x7f2384073360 2023-11-24T07:58:46.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.703+0000 7f23837fe700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f237006aed0 0x7f237006d390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23840737a0 tx=0x7f237801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.703+0000 7f5ecde5f700 1 Processor -- start 2023-11-24T07:58:46.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.704+0000 7f5ecde5f700 1 -- start start 2023-11-24T07:58:46.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ecde5f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ec80b2fc0 0x7f5ec814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ecde5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ecde5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec815dfc0 0x7f5ec80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ecde5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec815f3d0 con 0x7f5ec80d44e0 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ecde5f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec815f6b0 con 0x7f5ec815dfc0 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ecde5f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec815f990 con 0x7f5ec80b2fc0 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec77fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ec80b2fc0 0x7f5ec814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56824/0 (socket says 172.21.15.32:56824) 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 -- 172.21.15.32:0/3874801264 learned_addr learned my addr 172.21.15.32:0/3874801264 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 -- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ec80b2fc0 msgr2=0x7f5ec814b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ec80b2fc0 0x7f5ec814b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 -- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec815dfc0 msgr2=0x7f5ec80ca600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.705+0000 7f5ec6ffd700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec815dfc0 0x7f5ec80ca600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.706+0000 7f5ec6ffd700 1 -- 172.21.15.32:0/3874801264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009a70 con 0x7f5ec80d44e0 2023-11-24T07:58:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.706+0000 7f5ec6ffd700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80ca0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b450 tx=0x7f5ebc00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.706+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2374054590 con 0x7f2384073360 2023-11-24T07:58:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.708+0000 7f54b0a97700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 0x7f54ac0cd430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c4e0 tx=0x7f549c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8810054860 con 0x7f881c0b4090 2023-11-24T07:58:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0052d0 con 0x7f54ac0b3d40 2023-11-24T07:58:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c005430 con 0x7f54ac0b3d40 2023-11-24T07:58:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac1693b0 con 0x7f54ac0b3d40 2023-11-24T07:58:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0038a0 con 0x7f54ac0b3d40 2023-11-24T07:58:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac0ce130 con 0x7f54ac0b3d40 2023-11-24T07:58:46.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc011830 con 0x7f5ec80d44e0 2023-11-24T07:58:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc011990 con 0x7f5ec80d44e0 2023-11-24T07:58:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.709+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc011d40 con 0x7f5ec80d44e0 2023-11-24T07:58:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.711+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f549c003a00 con 0x7f54ac0b3d40 2023-11-24T07:58:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.712+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec815fc70 con 0x7f5ec80d44e0 2023-11-24T07:58:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.712+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec814b9b0 con 0x7f5ec80d44e0 2023-11-24T07:58:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.712+0000 7f54a1ffb700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f549006af20 0x7f549006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.712+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f549c088930 con 0x7f54ac0b3d40 2023-11-24T07:58:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.713+0000 7f54b1298700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f549006af20 0x7f549006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.713+0000 7f54b1298700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f549006af20 0x7f549006d3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0af520 tx=0x7f54a8010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.715+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ebc01a400 con 0x7f5ec80d44e0 2023-11-24T07:58:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.716+0000 7f5eaffff700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5eb406af20 0x7f5eb406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.716+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ebc089470 con 0x7f5ec80d44e0 2023-11-24T07:58:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.716+0000 7f5ec77fe700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5eb406af20 0x7f5eb406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.716+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8160190 con 0x7f5ec80d44e0 2023-11-24T07:58:46.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.717+0000 7f5ec77fe700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5eb406af20 0x7f5eb406d3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009bd0 tx=0x7f5eb80092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.718+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c0052e0 con 0x7f54ac0b3d40 2023-11-24T07:58:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.722+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ec8160190 con 0x7f5ec80d44e0 2023-11-24T07:58:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.723+0000 7f28b84fc700 1 Processor -- start 2023-11-24T07:58:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.723+0000 7f28b84fc700 1 -- start start 2023-11-24T07:58:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b84fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b84fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28b00ce660 0x7f28b00cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b84fc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b0162c40 con 0x7f28b0162860 2023-11-24T07:58:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b84fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b00cedf0 con 0x7f28b00ce660 2023-11-24T07:58:46.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b84fc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00cf0d0 con 0x7f28b0072df0 2023-11-24T07:58:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b6298700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f28b0162860 0x7f28b00726e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45298/0 (socket says 172.21.15.32:45298) 2023-11-24T07:58:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b6298700 1 -- 172.21.15.32:0/2959921862 learned_addr learned my addr 172.21.15.32:0/2959921862 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.725+0000 7f28b6a99700 1 --2- 172.21.15.32:0/2959921862 >> v2:172.21.15.32:3300/0 conn(0x7f28b00ce660 0x7f28b00cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.726+0000 7f28b6a99700 1 -- 172.21.15.32:0/2959921862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 msgr2=0x7f28b00b5180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:58:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.726+0000 7f28b6a99700 1 --2- 172.21.15.32:0/2959921862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00b5180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.726+0000 7f28b6a99700 1 -- 172.21.15.32:0/2959921862 >> v1:172.21.15.123:6789/0 conn(0x7f28b0162860 legacy=0x7f28b00726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.726+0000 7f28b6a99700 1 -- 172.21.15.32:0/2959921862 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b00cf3b0 con 0x7f28b00ce660 2023-11-24T07:58:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.726+0000 7f28b6a99700 1 --2- 172.21.15.32:0/2959921862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b00ce660 0x7f28b00cfe90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009d90 tx=0x7f28a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c2723016a4ddfa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.728+0000 7f28b5296700 1 -- 172.21.15.32:0/2959921862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a800e070 con 0x7f28b00ce660 2023-11-24T07:58:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.728+0000 7f28b5296700 1 -- 172.21.15.32:0/2959921862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a800bad0 con 0x7f28b00ce660 2023-11-24T07:58:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.728+0000 7f28b5296700 1 -- 172.21.15.32:0/2959921862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a800be80 con 0x7f28b00ce660 2023-11-24T07:58:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.729+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f549c053b00 con 0x7f54ac0b3d40 2023-11-24T07:58:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.729+0000 7f28b84fc700 1 -- 172.21.15.32:0/2959921862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b00ce660 msgr2=0x7f28b00cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.729+0000 7f28b84fc700 1 --2- 172.21.15.32:0/2959921862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b00ce660 0x7f28b00cfe90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009d90 tx=0x7f28a800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.730+0000 7f28b84fc700 1 -- 172.21.15.32:0/2959921862 shutdown_connections 2023-11-24T07:58:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.730+0000 7f28b84fc700 1 --2- 172.21.15.32:0/2959921862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b00ce660 0x7f28b00cfe90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.730+0000 7f28b84fc700 1 --2- 172.21.15.32:0/2959921862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.730+0000 7f28b84fc700 1 -- 172.21.15.32:0/2959921862 >> 172.21.15.32:0/2959921862 conn(0x7f28b00b57b0 msgr2=0x7f28b00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f28b84fc700 1 -- 172.21.15.32:0/2959921862 shutdown_connections 2023-11-24T07:58:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.735+0000 7f0e2f7ed700 1 Processor -- start 2023-11-24T07:58:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.735+0000 7f0e2f7ed700 1 -- start start 2023-11-24T07:58:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f0e2f7ed700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e280b3b50 0x7f0e280b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f0e2f7ed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e2806d4d0 0x7f0e2806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f0e2f7ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2806ced0 con 0x7f0e28163a80 2023-11-24T07:58:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f0e2f7ed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2806da70 con 0x7f0e2806d4d0 2023-11-24T07:58:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f0e2f7ed700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280b4470 con 0x7f0e280b3b50 2023-11-24T07:58:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.737+0000 7f0e2dd8a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e28163a80 0x7f0e28163f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58548/0 (socket says 172.21.15.32:58548) 2023-11-24T07:58:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.737+0000 7f0e2dd8a700 1 -- 172.21.15.32:0/3748221304 learned_addr learned my addr 172.21.15.32:0/3748221304 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:46.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.736+0000 7f28b84fc700 1 -- 172.21.15.32:0/2959921862 wait complete. 2023-11-24T07:58:46.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f28b84fc700 1 Processor -- start 2023-11-24T07:58:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005645741 0 0) 0x7f0e2806ced0 con 0x7f0e28163a80 2023-11-24T07:58:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e140035b0 con 0x7f0e28163a80 2023-11-24T07:58:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e2cd88700 1 --2- 172.21.15.32:0/3748221304 >> v2:172.21.15.32:3300/0 conn(0x7f0e2806d4d0 0x7f0e2806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e2d589700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e280b3b50 0x7f0e280b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307050966 0 0) 0x7f0e140035b0 con 0x7f0e28163a80 2023-11-24T07:58:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e280b3b50 msgr2=0x7f0e280b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e280b3b50 0x7f0e280b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 >> v2:172.21.15.32:3300/0 conn(0x7f0e2806d4d0 msgr2=0x7f0e2806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 --2- 172.21.15.32:0/3748221304 >> v2:172.21.15.32:3300/0 conn(0x7f0e2806d4d0 0x7f0e2806eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e281634e0 con 0x7f0e28163a80 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f0e200019f0 con 0x7f0e28163a80 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 >> v1:172.21.15.32:6789/0 conn(0x7f0e28163a80 legacy=0x7f0e28163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e14003a00 0x7f0e14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14004850 0x7f0e14004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e14005210 0x7f0e1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2806da70 con 0x7f0e14003a00 2023-11-24T07:58:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280b4470 con 0x7f0e14004850 2023-11-24T07:58:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1400c050 con 0x7f0e14005210 2023-11-24T07:58:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f0e2dd8a700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e14005210 0x7f0e1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.738+0000 7f28b84fc700 1 -- start start 2023-11-24T07:58:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b84fc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b84fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b00ce660 0x7f28b0153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b84fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 0x7f28b00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b84fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00ae3b0 con 0x7f28b0162860 2023-11-24T07:58:46.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2cd88700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14004850 0x7f0e14004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2d589700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e14003a00 0x7f0e14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2dd8a700 1 -- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14004850 msgr2=0x7f0e14004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2dd8a700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14004850 0x7f0e14004cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2dd8a700 1 -- 172.21.15.32:0/3748221304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e14003a00 msgr2=0x7f0e14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2dd8a700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e14003a00 0x7f0e14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2dd8a700 1 -- 172.21.15.32:0/3748221304 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e281634e0 con 0x7f0e14005210 2023-11-24T07:58:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b84fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00ae690 con 0x7f28b00ce660 2023-11-24T07:58:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b84fc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b00ae970 con 0x7f28b0072df0 2023-11-24T07:58:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b6a99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 0x7f28b00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b6a99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 0x7f28b00cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56850/0 (socket says 172.21.15.32:56850) 2023-11-24T07:58:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f28b6a99700 1 -- 172.21.15.32:0/1926237284 learned_addr learned my addr 172.21.15.32:0/1926237284 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.739+0000 7f0e2dd8a700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e14005210 0x7f0e1400ba10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000a8d0 tx=0x7f0e14005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ef021c732df46b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.740+0000 7f28b5a97700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b00ce660 0x7f28b0153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.740+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000a340 con 0x7f0e14005210 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.740+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e20012620 con 0x7f0e14005210 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.740+0000 7f0e1ffff700 1 -- 172.21.15.32:0/3748221304 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200129d0 con 0x7f0e14005210 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.741+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e14005210 msgr2=0x7f0e1400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.741+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/3748221304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e14005210 0x7f0e1400ba10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000a8d0 tx=0x7f0e14005880 comp rx=0 tx=0).stop 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.741+0000 7f0e2dd8a700 1 -- 172.21.15.32:0/3748221304 reap_dead start 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.746+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/3748221304 shutdown_connections 2023-11-24T07:58:46.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.746+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/3748221304 >> 172.21.15.32:0/3748221304 conn(0x7f0e280cfa30 msgr2=0x7f0e280cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.746+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/3748221304 shutdown_connections 2023-11-24T07:58:46.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.746+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/3748221304 wait complete. 2023-11-24T07:58:46.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.748+0000 7f0e2f7ed700 1 Processor -- start 2023-11-24T07:58:46.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f28b6298700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.748+0000 7f0e2f7ed700 1 -- start start 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.749+0000 7f0e2f7ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14005210 0x7f0e2814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.749+0000 7f0e2f7ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 0x7f0e28145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.749+0000 7f0e2f7ed700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e28143e90 0x7f0e28144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.749+0000 7f0e2f7ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e28147a80 con 0x7f0e28148b20 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.749+0000 7f0e2f7ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e28147d60 con 0x7f0e14005210 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.749+0000 7f0e2f7ed700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e28148040 con 0x7f0e28143e90 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 0x7f0e28145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 0x7f0e28145df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56856/0 (socket says 172.21.15.32:56856) 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 -- 172.21.15.32:0/170468551 learned_addr learned my addr 172.21.15.32:0/170468551 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2d589700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14005210 0x7f0e2814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 -- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e28143e90 msgr2=0x7f0e28144340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e28143e90 0x7f0e28144340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 -- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14005210 msgr2=0x7f0e2814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14005210 0x7f0e2814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2cd88700 1 -- 172.21.15.32:0/170468551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e28148320 con 0x7f0e28148b20 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.750+0000 7f0e2d589700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14005210 0x7f0e2814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:46.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f28b6a99700 1 -- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 msgr2=0x7f28b00af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f28b6a99700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f28b6a99700 1 -- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b00ce660 msgr2=0x7f28b0153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f28b6a99700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b00ce660 0x7f28b0153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f28b6a99700 1 -- 172.21.15.32:0/1926237284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8009a70 con 0x7f28b0162860 2023-11-24T07:58:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f0e2cd88700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 0x7f0e28145df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800f580 tx=0x7f0e1800f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f28b6a99700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 0x7f28b00cd4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28b006fd60 tx=0x7f28a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18014890 con 0x7f0e28148b20 2023-11-24T07:58:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1800c030 con 0x7f0e28148b20 2023-11-24T07:58:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18014bf0 con 0x7f0e28148b20 2023-11-24T07:58:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.752+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8004820 con 0x7f28b0162860 2023-11-24T07:58:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.752+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a801e430 con 0x7f28b0162860 2023-11-24T07:58:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.752+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a801e7e0 con 0x7f28b0162860 2023-11-24T07:58:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.752+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b00aec50 con 0x7f28b0162860 2023-11-24T07:58:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.752+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b00cd9e0 con 0x7f28b0162860 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e28148600 con 0x7f0e28148b20 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.751+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e28148fa0 con 0x7f0e28148b20 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.757+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e18014d50 con 0x7f0e28148b20 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.757+0000 7f0e1dffb700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0e1406c870 0x7f0e1406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.757+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e18089ae0 con 0x7f0e28148b20 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.757+0000 7f0e2d589700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0e1406c870 0x7f0e1406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.758+0000 7f0e2d589700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0e1406c870 0x7f0e1406ed30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e24006b20 tx=0x7f0e24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.754+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b00cdba0 con 0x7f28b0162860 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.755+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28a801e940 con 0x7f28b0162860 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.755+0000 7f28a6ffd700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f289406af20 0x7f289406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.756+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28a808bed0 con 0x7f28b0162860 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.756+0000 7f28b6298700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f289406af20 0x7f289406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.757+0000 7f28b6298700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f289406af20 0x7f289406d3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f28b00afbf0 tx=0x7f28ac007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.761+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e080052e0 con 0x7f0e28148b20 2023-11-24T07:58:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.762+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28a8057210 con 0x7f28b0162860 2023-11-24T07:58:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.771+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e18054b70 con 0x7f0e28148b20 2023-11-24T07:58:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.777+0000 7fba71e86700 1 Processor -- start 2023-11-24T07:58:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.777+0000 7fba71e86700 1 -- start start 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba71e86700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fba6c15d6a0 0x7fba6c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba71e86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba6c0b4090 0x7fba6c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba71e86700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c071a10 0x7fba6c06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba71e86700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c072d60 con 0x7fba6c0b4090 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba71e86700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c06de80 con 0x7fba6c15d6a0 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba71e86700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0b4510 con 0x7fba6c071a10 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba6affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba6c0b4090 0x7fba6c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba6affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba6c0b4090 0x7fba6c0714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56866/0 (socket says 172.21.15.32:56866) 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.779+0000 7fba6affd700 1 -- 172.21.15.32:0/1457127266 learned_addr learned my addr 172.21.15.32:0/1457127266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6bfff700 1 --2- 172.21.15.32:0/1457127266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c071a10 0x7fba6c06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6b7fe700 1 --2- 172.21.15.32:0/1457127266 >> v2:172.21.15.123:3300/0 conn(0x7fba6c15d6a0 0x7fba6c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6affd700 1 -- 172.21.15.32:0/1457127266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c071a10 msgr2=0x7fba6c06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6affd700 1 --2- 172.21.15.32:0/1457127266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c071a10 0x7fba6c06cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6affd700 1 -- 172.21.15.32:0/1457127266 >> v2:172.21.15.123:3300/0 conn(0x7fba6c15d6a0 msgr2=0x7fba6c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6affd700 1 --2- 172.21.15.32:0/1457127266 >> v2:172.21.15.123:3300/0 conn(0x7fba6c15d6a0 0x7fba6c0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6affd700 1 -- 172.21.15.32:0/1457127266 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0720e0 con 0x7fba6c0b4090 2023-11-24T07:58:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.780+0000 7fba6affd700 1 --2- 172.21.15.32:0/1457127266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c0b4090 0x7fba6c0714d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cfa00 tx=0x7fba6000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf0ce414d288857 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.784+0000 7fba6a7fc700 1 -- 172.21.15.32:0/1457127266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60010070 con 0x7fba6c0b4090 2023-11-24T07:58:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.784+0000 7fba6a7fc700 1 -- 172.21.15.32:0/1457127266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000d990 con 0x7fba6c0b4090 2023-11-24T07:58:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.784+0000 7fba6a7fc700 1 -- 172.21.15.32:0/1457127266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000dd00 con 0x7fba6c0b4090 2023-11-24T07:58:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 -- 172.21.15.32:0/1457127266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c0b4090 msgr2=0x7fba6c0714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 --2- 172.21.15.32:0/1457127266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c0b4090 0x7fba6c0714d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cfa00 tx=0x7fba6000d330 comp rx=0 tx=0).stop 2023-11-24T07:58:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 -- 172.21.15.32:0/1457127266 shutdown_connections 2023-11-24T07:58:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 --2- 172.21.15.32:0/1457127266 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c071a10 0x7fba6c06cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 --2- 172.21.15.32:0/1457127266 >> v2:172.21.15.123:3300/0 conn(0x7fba6c15d6a0 0x7fba6c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 --2- 172.21.15.32:0/1457127266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c0b4090 0x7fba6c0714d0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cfa00 tx=0x7fba6000d330 comp rx=0 tx=0).stop 2023-11-24T07:58:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.785+0000 7fba71e86700 1 -- 172.21.15.32:0/1457127266 >> 172.21.15.32:0/1457127266 conn(0x7fba6c0cfa30 msgr2=0x7fba6c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.789+0000 7fba71e86700 1 -- 172.21.15.32:0/1457127266 shutdown_connections 2023-11-24T07:58:46.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.789+0000 7fba71e86700 1 -- 172.21.15.32:0/1457127266 wait complete. 2023-11-24T07:58:46.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.791+0000 7fba71e86700 1 Processor -- start 2023-11-24T07:58:46.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.792+0000 7fba71e86700 1 -- start start 2023-11-24T07:58:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.793+0000 7fba71e86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c071a10 0x7fba6c0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.793+0000 7fba71e86700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c15d6a0 0x7fba6c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.793+0000 7fba71e86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6c13bb30 0x7fba6c0abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.793+0000 7fba71e86700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0ab1b0 con 0x7fba6c071a10 2023-11-24T07:58:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.793+0000 7fba71e86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0aa830 con 0x7fba6c13bb30 2023-11-24T07:58:46.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.793+0000 7fba71e86700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0aaaf0 con 0x7fba6c15d6a0 2023-11-24T07:58:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.794+0000 7fba6affd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c15d6a0 0x7fba6c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.794+0000 7fba6bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6c13bb30 0x7fba6c0abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.794+0000 7fba6bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6c13bb30 0x7fba6c0abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:41020/0 (socket says 172.21.15.32:41020) 2023-11-24T07:58:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.794+0000 7fba6bfff700 1 -- 172.21.15.32:0/2181544096 learned_addr learned my addr 172.21.15.32:0/2181544096 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.794+0000 7fba6b7fe700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c071a10 0x7fba6c0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.795+0000 7fba6b7fe700 1 -- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c15d6a0 msgr2=0x7fba6c0a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.795+0000 7fba6b7fe700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c15d6a0 0x7fba6c0a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.795+0000 7fba6b7fe700 1 -- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6c13bb30 msgr2=0x7fba6c0abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.795+0000 7fba6b7fe700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6c13bb30 0x7fba6c0abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.795+0000 7fba6b7fe700 1 -- 172.21.15.32:0/2181544096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600085d0 con 0x7fba6c071a10 2023-11-24T07:58:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.795+0000 7fba6b7fe700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c071a10 0x7fba6c0aee60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00e900 tx=0x7fba5c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.796+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00ee50 con 0x7fba6c071a10 2023-11-24T07:58:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.796+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c00c850 con 0x7fba6c071a10 2023-11-24T07:58:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.796+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00cbc0 con 0x7fba6c071a10 2023-11-24T07:58:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.796+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6c0aadd0 con 0x7fba6c071a10 2023-11-24T07:58:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.796+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6c13c180 con 0x7fba6c071a10 2023-11-24T07:58:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.802+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba5c00cd20 con 0x7fba6c071a10 2023-11-24T07:58:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.803+0000 7fba53fff700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fba5806afa0 0x7fba5806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.803+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba5c0895b0 con 0x7fba6c071a10 2023-11-24T07:58:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.804+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4c0052e0 con 0x7fba6c071a10 2023-11-24T07:58:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.808+0000 7fba6affd700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fba5806afa0 0x7fba5806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.814+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba5c054640 con 0x7fba6c071a10 2023-11-24T07:58:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:46.814+0000 7fba6affd700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fba5806afa0 0x7fba5806d460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cfa00 tx=0x7fba60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.099+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f84a0005590 con 0x7f84b815dde0 2023-11-24T07:58:47.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.104+0000 7f84ae7fc700 1 -- 172.21.15.32:0/58546691 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f84b001b020 con 0x7f84b815dde0 2023-11-24T07:58:47.111 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-11-24T07:58:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f84a406c800 msgr2=0x7f84a406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f84a406c800 0x7f84a406ecc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000fb0 tx=0x7f84a8009040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 msgr2=0x7f84b81637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 0x7f84b81637c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f84b000ec90 tx=0x7f84b0004b40 comp rx=0 tx=0).stop 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 shutdown_connections 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f84a406c800 0x7f84a406ecc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84b815e1c0 0x7f84b815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84b815dde0 0x7f84b81637c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 --2- 172.21.15.32:0/58546691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b80cd1c0 0x7f84b80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.110+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 >> 172.21.15.32:0/58546691 conn(0x7f84b80cfa30 msgr2=0x7f84b8071140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.111+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 shutdown_connections 2023-11-24T07:58:47.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.111+0000 7f84bfab2700 1 -- 172.21.15.32:0/58546691 wait complete. 2023-11-24T07:58:47.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.113+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2364005590 con 0x7f2384073360 2023-11-24T07:58:47.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.117+0000 7f2380ff9700 1 -- 172.21.15.32:0/1857481139 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2374021070 con 0x7f2384073360 2023-11-24T07:58:47.125 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.124+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f237006aed0 msgr2=0x7f237006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.124+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f237006aed0 0x7f237006d390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23840737a0 tx=0x7f237801c040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.124+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 msgr2=0x7f2384099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.124+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 0x7f2384099a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f237400e870 tx=0x7f237400eb80 comp rx=0 tx=0).stop 2023-11-24T07:58:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f548c005860 con 0x7f54ac0b3d40 2023-11-24T07:58:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 shutdown_connections 2023-11-24T07:58:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f237006aed0 0x7f237006d390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2384162750 0x7f23840c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2384073360 0x7f2384099a50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 --2- 172.21.15.32:0/1857481139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23840c3820 0x7f23841ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 >> 172.21.15.32:0/1857481139 conn(0x7f23840d0230 msgr2=0x7f23840c3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f54a1ffb700 1 -- 172.21.15.32:0/1047577748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f549c017020 con 0x7f54ac0b3d40 2023-11-24T07:58:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 shutdown_connections 2023-11-24T07:58:47.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.125+0000 7f238a5b5700 1 -- 172.21.15.32:0/1857481139 wait complete. 2023-11-24T07:58:47.129 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:47.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-11-24T07:58:47.132 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.131+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f549006af20 msgr2=0x7f549006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.131+0000 7f54b34fc700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f549006af20 0x7f549006d3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0af520 tx=0x7f54a8010000 comp rx=0 tx=0).stop 2023-11-24T07:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.131+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 msgr2=0x7f54ac0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.131+0000 7f54b34fc700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 0x7f54ac0cd430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c4e0 tx=0x7f549c00c7f0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5ec8160190 con 0x7f5ec80d44e0 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 shutdown_connections 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f549006af20 0x7f549006d3e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac140680 0x7f54ac0aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54ac071470 0x7f54ac15fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 --2- 172.21.15.32:0/1047577748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54ac0b3d40 0x7f54ac0cd430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 >> 172.21.15.32:0/1047577748 conn(0x7f54ac0b1740 msgr2=0x7f54ac0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 shutdown_connections 2023-11-24T07:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.132+0000 7f54b34fc700 1 -- 172.21.15.32:0/1047577748 wait complete. 2023-11-24T07:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.133+0000 7f5eaffff700 1 -- 172.21.15.32:0/3874801264 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ec8160190 con 0x7f5ec80d44e0 2023-11-24T07:58:47.136 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.137+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f87fc005860 con 0x7f881c0b4090 2023-11-24T07:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5eb406af20 msgr2=0x7f5eb406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5eb406af20 0x7f5eb406d3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009bd0 tx=0x7f5eb80092c0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 msgr2=0x7f5ec80ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80ca0c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b450 tx=0x7f5ebc00b760 comp rx=0 tx=0).stop 2023-11-24T07:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f8803fff700 1 -- 172.21.15.32:0/3976174439 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8810057e80 con 0x7f881c0b4090 2023-11-24T07:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 shutdown_connections 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5eb406af20 0x7f5eb406d3e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec815dfc0 0x7f5ec80ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ec80b2fc0 0x7f5ec814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 --2- 172.21.15.32:0/3874801264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80d44e0 0x7f5ec80ca0c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 >> 172.21.15.32:0/3874801264 conn(0x7f5ec806b410 msgr2=0x7f5ec80ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 shutdown_connections 2023-11-24T07:58:47.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.138+0000 7f5ecde5f700 1 -- 172.21.15.32:0/3874801264 wait complete. 2023-11-24T07:58:47.142 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.143+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f880806af20 msgr2=0x7f880806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.143+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f880806af20 0x7f880806d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b370 tx=0x7f880c011000 comp rx=0 tx=0).stop 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.143+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 msgr2=0x7f881c15b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.143+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 0x7f881c15b2d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f881000d8d0 tx=0x7f881000dbe0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.144+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 shutdown_connections 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.144+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f880806af20 0x7f880806d3e0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b370 tx=0x7f880c011000 comp rx=0 tx=0).stop 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.144+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f881c15d6a0 0x7f881c0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.144+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f881c071a10 0x7f881c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.144+0000 7f8821cc1700 1 --2- 172.21.15.32:0/3976174439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f881c0b4090 0x7f881c15b2d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.144+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 >> 172.21.15.32:0/3976174439 conn(0x7f881c0cfa30 msgr2=0x7f881c15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.145+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 shutdown_connections 2023-11-24T07:58:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.145+0000 7f8821cc1700 1 -- 172.21.15.32:0/3976174439 wait complete. 2023-11-24T07:58:47.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-11-24T07:58:47.148 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2023-11-24T07:58:47.153 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-11-24T07:58:47.159 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-11-24T07:58:47.163 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.178+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f28b00af2a0 con 0x7f28b0162860 2023-11-24T07:58:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.179+0000 7f28a6ffd700 1 -- 172.21.15.32:0/1926237284 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f28a805a830 con 0x7f28b0162860 2023-11-24T07:58:47.181 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-11-24T07:58:47.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f289406af20 msgr2=0x7f289406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f289406af20 0x7f289406d3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f28b00afbf0 tx=0x7f28ac007d70 comp rx=0 tx=0).stop 2023-11-24T07:58:47.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 msgr2=0x7f28b00cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 0x7f28b00cd4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28b006fd60 tx=0x7f28a8004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 shutdown_connections 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f289406af20 0x7f289406d3e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b0162860 0x7f28b00cd4a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f28b0072df0 0x7f28b00af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 --2- 172.21.15.32:0/1926237284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b00ce660 0x7f28b0153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 >> 172.21.15.32:0/1926237284 conn(0x7f28b00b57b0 msgr2=0x7f28b00b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 shutdown_connections 2023-11-24T07:58:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.182+0000 7f28b84fc700 1 -- 172.21.15.32:0/1926237284 wait complete. 2023-11-24T07:58:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.188+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fba4c005590 con 0x7fba6c071a10 2023-11-24T07:58:47.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.188+0000 7fba53fff700 1 -- 172.21.15.32:0/2181544096 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba5c021020 con 0x7fba6c071a10 2023-11-24T07:58:47.190 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fba5806afa0 msgr2=0x7fba5806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fba5806afa0 0x7fba5806d460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cfa00 tx=0x7fba60008040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0e08005590 con 0x7f0e28148b20 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c071a10 msgr2=0x7fba6c0aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c071a10 0x7fba6c0aee60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00e900 tx=0x7fba5c00ec10 comp rx=0 tx=0).stop 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 shutdown_connections 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fba5806afa0 0x7fba5806d460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6c13bb30 0x7fba6c0abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba6c071a10 0x7fba6c0aee60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 --2- 172.21.15.32:0/2181544096 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba6c15d6a0 0x7fba6c0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.190+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 >> 172.21.15.32:0/2181544096 conn(0x7fba6c0cfa30 msgr2=0x7fba6c0b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.191+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 shutdown_connections 2023-11-24T07:58:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.191+0000 7fba71e86700 1 -- 172.21.15.32:0/2181544096 wait complete. 2023-11-24T07:58:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.191+0000 7f0e1dffb700 1 -- 172.21.15.32:0/170468551 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e18058190 con 0x7f0e28148b20 2023-11-24T07:58:47.194 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0e1406c870 msgr2=0x7f0e1406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0e1406c870 0x7f0e1406ed30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e24006b20 tx=0x7f0e24008040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 msgr2=0x7f0e28145df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 0x7f0e28145df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800f580 tx=0x7f0e1800f890 comp rx=0 tx=0).stop 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 shutdown_connections 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0e1406c870 0x7f0e1406ed30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e28143e90 0x7f0e28144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14005210 0x7f0e2814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 --2- 172.21.15.32:0/170468551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e28148b20 0x7f0e28145df0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 >> 172.21.15.32:0/170468551 conn(0x7f0e280cfa30 msgr2=0x7f0e28162520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 shutdown_connections 2023-11-24T07:58:47.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.195+0000 7f0e2f7ed700 1 -- 172.21.15.32:0/170468551 wait complete. 2023-11-24T07:58:47.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2023-11-24T07:58:47.202 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-11-24T07:58:47.206 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-11-24T07:58:47.211 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:47.211 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T07:58:47.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.375+0000 7f37a341d700 1 Processor -- start 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.375+0000 7f37a341d700 1 -- start start 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a341d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a341d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f379c0d0040 0x7f379c0d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a341d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c0d09d0 con 0x7f379c15c1e0 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a341d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c0b00e0 con 0x7f379c0d0040 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a341d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0af200 con 0x7f379c15be00 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f379c0d0040 0x7f379c0d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a09b8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f379c15c1e0 0x7f379c0aeaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58588/0 (socket says 172.21.15.32:58588) 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f379c0d0040 0x7f379c0d0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56878/0 (socket says 172.21.15.32:56878) 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a09b8700 1 -- 172.21.15.32:0/2689508914 learned_addr learned my addr 172.21.15.32:0/2689508914 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a11b9700 1 --2- 172.21.15.32:0/2689508914 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 -- 172.21.15.32:0/2689508914 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 msgr2=0x7f379c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 --2- 172.21.15.32:0/2689508914 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 -- 172.21.15.32:0/2689508914 >> v1:172.21.15.32:6789/0 conn(0x7f379c15c1e0 legacy=0x7f379c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f3793fff700 1 -- 172.21.15.32:0/2689508914 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82457467 0 0) 0x7f378c001110 con 0x7f379c15c1e0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 -- 172.21.15.32:0/2689508914 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0af4d0 con 0x7f379c0d0040 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.376+0000 7f37a19ba700 1 --2- 172.21.15.32:0/2689508914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c0d0490 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3794009d90 tx=0x7f379400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ba764ca072adcb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f3793fff700 1 -- 172.21.15.32:0/2689508914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400e070 con 0x7f379c0d0040 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f3793fff700 1 -- 172.21.15.32:0/2689508914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379400bb20 con 0x7f379c0d0040 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f3793fff700 1 -- 172.21.15.32:0/2689508914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794012430 con 0x7f379c0d0040 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 -- 172.21.15.32:0/2689508914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 msgr2=0x7f379c0d0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 --2- 172.21.15.32:0/2689508914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c0d0490 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3794009d90 tx=0x7f379400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 -- 172.21.15.32:0/2689508914 shutdown_connections 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 --2- 172.21.15.32:0/2689508914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c0d0490 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 --2- 172.21.15.32:0/2689508914 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 -- 172.21.15.32:0/2689508914 >> 172.21.15.32:0/2689508914 conn(0x7f379c09aa00 msgr2=0x7f379c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 -- 172.21.15.32:0/2689508914 shutdown_connections 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.377+0000 7f37a341d700 1 -- 172.21.15.32:0/2689508914 wait complete. 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.378+0000 7f37a341d700 1 Processor -- start 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.378+0000 7f37a341d700 1 -- start start 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a341d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a341d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a341d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f379c15c1e0 0x7f379c150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a341d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c06d020 con 0x7f379c0d0040 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a341d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0cb500 con 0x7f379c15c1e0 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a341d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c06c8f0 con 0x7f379c15be00 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a11b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c06cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a11b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c06cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56880/0 (socket says 172.21.15.32:56880) 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a11b9700 1 -- 172.21.15.32:0/520910709 learned_addr learned my addr 172.21.15.32:0/520910709 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.379+0000 7f37a19ba700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f379c15c1e0 0x7f379c150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a09b8700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a11b9700 1 -- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 msgr2=0x7f379c0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a11b9700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a11b9700 1 -- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f379c15c1e0 msgr2=0x7f379c150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a11b9700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f379c15c1e0 0x7f379c150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a11b9700 1 -- 172.21.15.32:0/520910709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3794009a70 con 0x7f379c0d0040 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a19ba700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f379c15c1e0 0x7f379c150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a09b8700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a11b9700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c06cae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f379800e9e0 tx=0x7f379800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798009710 con 0x7f379c0d0040 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800c850 con 0x7f379c0d0040 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c06bde0 con 0x7f379c0d0040 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800cbc0 con 0x7f379c0d0040 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.380+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c0cafe0 con 0x7f379c0d0040 2023-11-24T07:58:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.381+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c06c440 con 0x7f379c0d0040 2023-11-24T07:58:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.382+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f379800cd20 con 0x7f379c0d0040 2023-11-24T07:58:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.382+0000 7f3791ffb700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f378006aed0 0x7f378006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.382+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3798089620 con 0x7f379c0d0040 2023-11-24T07:58:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.382+0000 7f37a09b8700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f378006aed0 0x7f378006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.383+0000 7f37a09b8700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f378006aed0 0x7f378006d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f378c007fd0 tx=0x7f378c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.386+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3798054760 con 0x7f379c0d0040 2023-11-24T07:58:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.585+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f379c0afd70 con 0x7f378006aed0 2023-11-24T07:58:47.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.592+0000 7f3791ffb700 1 -- 172.21.15.32:0/520910709 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264533 (secure 0 0 0) 0x7f378c001cd0 con 0x7f378006aed0 2023-11-24T07:58:47.598 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.595+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f378006aed0 msgr2=0x7f378006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.595+0000 7f37a341d700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f378006aed0 0x7f378006d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f378c007fd0 tx=0x7f378c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.595+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 msgr2=0x7f379c06cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.595+0000 7f37a341d700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c06cae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f379800e9e0 tx=0x7f379800eda0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 shutdown_connections 2023-11-24T07:58:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f378006aed0 0x7f378006d390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f379c15c1e0 0x7f379c150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c0d0040 0x7f379c06cae0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 --2- 172.21.15.32:0/520910709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f379c15be00 0x7f379c0d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 >> 172.21.15.32:0/520910709 conn(0x7f379c09aa00 msgr2=0x7f379c15b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 shutdown_connections 2023-11-24T07:58:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.596+0000 7f37a341d700 1 -- 172.21.15.32:0/520910709 wait complete. 2023-11-24T07:58:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T07:58:47.610 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T07:58:46.531391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219104,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755616,"statfs":{"total":773094113280,"available":772869750784,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210697+0000","last_change":"2023-11-24T07:58:38.673717+0000","last_active":"2023-11-24T07:58:40.210697+0000","last_peered":"2023-11-24T07:58:40.210697+0000","last_clean":"2023-11-24T07:58:40.210697+0000","last_became_active":"2023-11-24T07:58:38.673289+0000","last_became_peered":"2023-11-24T07:58:38.673289+0000","last_unstale":"2023-11-24T07:58:40.210697+0000","last_undegraded":"2023-11-24T07:58:40.210697+0000","last_fullsized":"2023-11-24T07:58:40.210697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:16:17.544102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209942+0000","last_change":"2023-11-24T07:58:38.666572+0000","last_active":"2023-11-24T07:58:40.209942+0000","last_peered":"2023-11-24T07:58:40.209942+0000","last_clean":"2023-11-24T07:58:40.209942+0000","last_became_active":"2023-11-24T07:58:38.666220+0000","last_became_peered":"2023-11-24T07:58:38.666220+0000","last_unstale":"2023-11-24T07:58:40.209942+0000","last_undegraded":"2023-11-24T07:58:40.209942+0000","last_fullsized":"2023-11-24T07:58:40.209942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670074+0000","last_change":"2023-11-24T07:58:38.664139+0000","last_active":"2023-11-24T07:58:39.670074+0000","last_peered":"2023-11-24T07:58:39.670074+0000","last_clean":"2023-11-24T07:58:39.670074+0000","last_became_active":"2023-11-24T07:58:38.660601+0000","last_became_peered":"2023-11-24T07:58:38.660601+0000","last_unstale":"2023-11-24T07:58:39.670074+0000","last_undegraded":"2023-11-24T07:58:39.670074+0000","last_fullsized":"2023-11-24T07:58:39.670074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662796+0000","last_change":"2023-11-24T07:58:38.670834+0000","last_active":"2023-11-24T07:58:39.662796+0000","last_peered":"2023-11-24T07:58:39.662796+0000","last_clean":"2023-11-24T07:58:39.662796+0000","last_became_active":"2023-11-24T07:58:38.670567+0000","last_became_peered":"2023-11-24T07:58:38.670567+0000","last_unstale":"2023-11-24T07:58:39.662796+0000","last_undegraded":"2023-11-24T07:58:39.662796+0000","last_fullsized":"2023-11-24T07:58:39.662796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210112+0000","last_change":"2023-11-24T07:58:38.657900+0000","last_active":"2023-11-24T07:58:40.210112+0000","last_peered":"2023-11-24T07:58:40.210112+0000","last_clean":"2023-11-24T07:58:40.210112+0000","last_became_active":"2023-11-24T07:58:38.657374+0000","last_became_peered":"2023-11-24T07:58:38.657374+0000","last_unstale":"2023-11-24T07:58:40.210112+0000","last_undegraded":"2023-11-24T07:58:40.210112+0000","last_fullsized":"2023-11-24T07:58:40.210112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662085+0000","last_change":"2023-11-24T07:58:38.668973+0000","last_active":"2023-11-24T07:58:39.662085+0000","last_peered":"2023-11-24T07:58:39.662085+0000","last_clean":"2023-11-24T07:58:39.662085+0000","last_became_active":"2023-11-24T07:58:38.668660+0000","last_became_peered":"2023-11-24T07:58:38.668660+0000","last_unstale":"2023-11-24T07:58:39.662085+0000","last_undegraded":"2023-11-24T07:58:39.662085+0000","last_fullsized":"2023-11-24T07:58:39.662085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209727+0000","last_change":"2023-11-24T07:58:38.670604+0000","last_active":"2023-11-24T07:58:40.209727+0000","last_peered":"2023-11-24T07:58:40.209727+0000","last_clean":"2023-11-24T07:58:40.209727+0000","last_became_active":"2023-11-24T07:58:38.670304+0000","last_became_peered":"2023-11-24T07:58:38.670304+0000","last_unstale":"2023-11-24T07:58:40.209727+0000","last_undegraded":"2023-11-24T07:58:40.209727+0000","last_fullsized":"2023-11-24T07:58:40.209727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655626+0000","last_change":"2023-11-24T07:58:38.671094+0000","last_active":"2023-11-24T07:58:39.655626+0000","last_peered":"2023-11-24T07:58:39.655626+0000","last_clean":"2023-11-24T07:58:39.655626+0000","last_became_active":"2023-11-24T07:58:38.670534+0000","last_became_peered":"2023-11-24T07:58:38.670534+0000","last_unstale":"2023-11-24T07:58:39.655626+0000","last_undegraded":"2023-11-24T07:58:39.655626+0000","last_fullsized":"2023-11-24T07:58:39.655626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210625+0000","last_change":"2023-11-24T07:58:38.673877+0000","last_active":"2023-11-24T07:58:40.210625+0000","last_peered":"2023-11-24T07:58:40.210625+0000","last_clean":"2023-11-24T07:58:40.210625+0000","last_became_active":"2023-11-24T07:58:38.673449+0000","last_became_peered":"2023-11-24T07:58:38.673449+0000","last_unstale":"2023-11-24T07:58:40.210625+0000","last_undegraded":"2023-11-24T07:58:40.210625+0000","last_fullsized":"2023-11-24T07:58:40.210625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661371+0000","last_change":"2023-11-24T07:58:38.667090+0000","last_active":"2023-11-24T07:58:39.661371+0000","last_peered":"2023-11-24T07:58:39.661371+0000","last_clean":"2023-11-24T07:58:39.661371+0000","last_became_active":"2023-11-24T07:58:38.665486+0000","last_became_peered":"2023-11-24T07:58:38.665486+0000","last_unstale":"2023-11-24T07:58:39.661371+0000","last_undegraded":"2023-11-24T07:58:39.661371+0000","last_fullsized":"2023-11-24T07:58:39.661371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661214+0000","last_change":"2023-11-24T07:58:38.666051+0000","last_active":"2023-11-24T07:58:39.661214+0000","last_peered":"2023-11-24T07:58:39.661214+0000","last_clean":"2023-11-24T07:58:39.661214+0000","last_became_active":"2023-11-24T07:58:38.665574+0000","last_became_peered":"2023-11-24T07:58:38.665574+0000","last_unstale":"2023-11-24T07:58:39.661214+0000","last_undegraded":"2023-11-24T07:58:39.661214+0000","last_fullsized":"2023-11-24T07:58:39.661214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210541+0000","last_change":"2023-11-24T07:58:38.667776+0000","last_active":"2023-11-24T07:58:40.210541+0000","last_peered":"2023-11-24T07:58:40.210541+0000","last_clean":"2023-11-24T07:58:40.210541+0000","last_became_active":"2023-11-24T07:58:38.667513+0000","last_became_peered":"2023-11-24T07:58:38.667513+0000","last_unstale":"2023-11-24T07:58:40.210541+0000","last_undegraded":"2023-11-24T07:58:40.210541+0000","last_fullsized":"2023-11-24T07:58:40.210541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655517+0000","last_change":"2023-11-24T07:58:38.672201+0000","last_active":"2023-11-24T07:58:39.655517+0000","last_peered":"2023-11-24T07:58:39.655517+0000","last_clean":"2023-11-24T07:58:39.655517+0000","last_became_active":"2023-11-24T07:58:38.671869+0000","last_became_peered":"2023-11-24T07:58:38.671869+0000","last_unstale":"2023-11-24T07:58:39.655517+0000","last_undegraded":"2023-11-24T07:58:39.655517+0000","last_fullsized":"2023-11-24T07:58:39.655517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670156+0000","last_change":"2023-11-24T07:58:38.662685+0000","last_active":"2023-11-24T07:58:39.670156+0000","last_peered":"2023-11-24T07:58:39.670156+0000","last_clean":"2023-11-24T07:58:39.670156+0000","last_became_active":"2023-11-24T07:58:38.660043+0000","last_became_peered":"2023-11-24T07:58:38.660043+0000","last_unstale":"2023-11-24T07:58:39.670156+0000","last_undegraded":"2023-11-24T07:58:39.670156+0000","last_fullsized":"2023-11-24T07:58:39.670156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209459+0000","last_change":"2023-11-24T07:58:38.667653+0000","last_active":"2023-11-24T07:58:40.209459+0000","last_peered":"2023-11-24T07:58:40.209459+0000","last_clean":"2023-11-24T07:58:40.209459+0000","last_became_active":"2023-11-24T07:58:38.667388+0000","last_became_peered":"2023-11-24T07:58:38.667388+0000","last_unstale":"2023-11-24T07:58:40.209459+0000","last_undegraded":"2023-11-24T07:58:40.209459+0000","last_fullsized":"2023-11-24T07:58:40.209459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661870+0000","last_change":"2023-11-24T07:58:38.669289+0000","last_active":"2023-11-24T07:58:39.661870+0000","last_peered":"2023-11-24T07:58:39.661870+0000","last_clean":"2023-11-24T07:58:39.661870+0000","last_became_active":"2023-11-24T07:58:38.668715+0000","last_became_peered":"2023-11-24T07:58:38.668715+0000","last_unstale":"2023-11-24T07:58:39.661870+0000","last_undegraded":"2023-11-24T07:58:39.661870+0000","last_fullsized":"2023-11-24T07:58:39.661870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662528+0000","last_change":"2023-11-24T07:58:38.674765+0000","last_active":"2023-11-24T07:58:39.662528+0000","last_peered":"2023-11-24T07:58:39.662528+0000","last_clean":"2023-11-24T07:58:39.662528+0000","last_became_active":"2023-11-24T07:58:38.674479+0000","last_became_peered":"2023-11-24T07:58:38.674479+0000","last_unstale":"2023-11-24T07:58:39.662528+0000","last_undegraded":"2023-11-24T07:58:39.662528+0000","last_fullsized":"2023-11-24T07:58:39.662528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210293+0000","last_change":"2023-11-24T07:58:38.667446+0000","last_active":"2023-11-24T07:58:40.210293+0000","last_peered":"2023-11-24T07:58:40.210293+0000","last_clean":"2023-11-24T07:58:40.210293+0000","last_became_active":"2023-11-24T07:58:38.667164+0000","last_became_peered":"2023-11-24T07:58:38.667164+0000","last_unstale":"2023-11-24T07:58:40.210293+0000","last_undegraded":"2023-11-24T07:58:40.210293+0000","last_fullsized":"2023-11-24T07:58:40.210293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210359+0000","last_change":"2023-11-24T07:58:38.663509+0000","last_active":"2023-11-24T07:58:40.210359+0000","last_peered":"2023-11-24T07:58:40.210359+0000","last_clean":"2023-11-24T07:58:40.210359+0000","last_became_active":"2023-11-24T07:58:38.661576+0000","last_became_peered":"2023-11-24T07:58:38.661576+0000","last_unstale":"2023-11-24T07:58:40.210359+0000","last_undegraded":"2023-11-24T07:58:40.210359+0000","last_fullsized":"2023-11-24T07:58:40.210359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669966+0000","last_change":"2023-11-24T07:58:37.664997+0000","last_active":"2023-11-24T07:58:39.669966+0000","last_peered":"2023-11-24T07:58:39.669966+0000","last_clean":"2023-11-24T07:58:39.669966+0000","last_became_active":"2023-11-24T07:58:37.664509+0000","last_became_peered":"2023-11-24T07:58:37.664509+0000","last_unstale":"2023-11-24T07:58:39.669966+0000","last_undegraded":"2023-11-24T07:58:39.669966+0000","last_fullsized":"2023-11-24T07:58:39.669966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656179+0000","last_change":"2023-11-24T07:58:38.671945+0000","last_active":"2023-11-24T07:58:39.656179+0000","last_peered":"2023-11-24T07:58:39.656179+0000","last_clean":"2023-11-24T07:58:39.656179+0000","last_became_active":"2023-11-24T07:58:38.671503+0000","last_became_peered":"2023-11-24T07:58:38.671503+0000","last_unstale":"2023-11-24T07:58:39.656179+0000","last_undegraded":"2023-11-24T07:58:39.656179+0000","last_fullsized":"2023-11-24T07:58:39.656179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656004+0000","last_change":"2023-11-24T07:58:37.664969+0000","last_active":"2023-11-24T07:58:39.656004+0000","last_peered":"2023-11-24T07:58:39.656004+0000","last_clean":"2023-11-24T07:58:39.656004+0000","last_became_active":"2023-11-24T07:58:37.664661+0000","last_became_peered":"2023-11-24T07:58:37.664661+0000","last_unstale":"2023-11-24T07:58:39.656004+0000","last_undegraded":"2023-11-24T07:58:39.656004+0000","last_fullsized":"2023-11-24T07:58:39.656004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670016+0000","last_change":"2023-11-24T07:58:38.670070+0000","last_active":"2023-11-24T07:58:39.670016+0000","last_peered":"2023-11-24T07:58:39.670016+0000","last_clean":"2023-11-24T07:58:39.670016+0000","last_became_active":"2023-11-24T07:58:38.669739+0000","last_became_peered":"2023-11-24T07:58:38.669739+0000","last_unstale":"2023-11-24T07:58:39.670016+0000","last_undegraded":"2023-11-24T07:58:39.670016+0000","last_fullsized":"2023-11-24T07:58:39.670016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:04.076457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210695+0000","last_change":"2023-11-24T07:58:37.663122+0000","last_active":"2023-11-24T07:58:40.210695+0000","last_peered":"2023-11-24T07:58:40.210695+0000","last_clean":"2023-11-24T07:58:40.210695+0000","last_became_active":"2023-11-24T07:58:37.662846+0000","last_became_peered":"2023-11-24T07:58:37.662846+0000","last_unstale":"2023-11-24T07:58:40.210695+0000","last_undegraded":"2023-11-24T07:58:40.210695+0000","last_fullsized":"2023-11-24T07:58:40.210695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:56.244472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662566+0000","last_change":"2023-11-24T07:58:38.669302+0000","last_active":"2023-11-24T07:58:39.662566+0000","last_peered":"2023-11-24T07:58:39.662566+0000","last_clean":"2023-11-24T07:58:39.662566+0000","last_became_active":"2023-11-24T07:58:38.669023+0000","last_became_peered":"2023-11-24T07:58:38.669023+0000","last_unstale":"2023-11-24T07:58:39.662566+0000","last_undegraded":"2023-11-24T07:58:39.662566+0000","last_fullsized":"2023-11-24T07:58:39.662566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210236+0000","last_change":"2023-11-24T07:58:37.659022+0000","last_active":"2023-11-24T07:58:40.210236+0000","last_peered":"2023-11-24T07:58:40.210236+0000","last_clean":"2023-11-24T07:58:40.210236+0000","last_became_active":"2023-11-24T07:58:37.658698+0000","last_became_peered":"2023-11-24T07:58:37.658698+0000","last_unstale":"2023-11-24T07:58:40.210236+0000","last_undegraded":"2023-11-24T07:58:40.210236+0000","last_fullsized":"2023-11-24T07:58:40.210236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662404+0000","last_change":"2023-11-24T07:58:38.669294+0000","last_active":"2023-11-24T07:58:39.662404+0000","last_peered":"2023-11-24T07:58:39.662404+0000","last_clean":"2023-11-24T07:58:39.662404+0000","last_became_active":"2023-11-24T07:58:38.668636+0000","last_became_peered":"2023-11-24T07:58:38.668636+0000","last_unstale":"2023-11-24T07:58:39.662404+0000","last_undegraded":"2023-11-24T07:58:39.662404+0000","last_fullsized":"2023-11-24T07:58:39.662404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655621+0000","last_change":"2023-11-24T07:58:37.655113+0000","last_active":"2023-11-24T07:58:39.655621+0000","last_peered":"2023-11-24T07:58:39.655621+0000","last_clean":"2023-11-24T07:58:39.655621+0000","last_became_active":"2023-11-24T07:58:37.654784+0000","last_became_peered":"2023-11-24T07:58:37.654784+0000","last_unstale":"2023-11-24T07:58:39.655621+0000","last_undegraded":"2023-11-24T07:58:39.655621+0000","last_fullsized":"2023-11-24T07:58:39.655621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209899+0000","last_change":"2023-11-24T07:58:38.653315+0000","last_active":"2023-11-24T07:58:40.209899+0000","last_peered":"2023-11-24T07:58:40.209899+0000","last_clean":"2023-11-24T07:58:40.209899+0000","last_became_active":"2023-11-24T07:58:38.653027+0000","last_became_peered":"2023-11-24T07:58:38.653027+0000","last_unstale":"2023-11-24T07:58:40.209899+0000","last_undegraded":"2023-11-24T07:58:40.209899+0000","last_fullsized":"2023-11-24T07:58:40.209899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:03.244592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660984+0000","last_change":"2023-11-24T07:58:37.660782+0000","last_active":"2023-11-24T07:58:39.660984+0000","last_peered":"2023-11-24T07:58:39.660984+0000","last_clean":"2023-11-24T07:58:39.660984+0000","last_became_active":"2023-11-24T07:58:37.660419+0000","last_became_peered":"2023-11-24T07:58:37.660419+0000","last_unstale":"2023-11-24T07:58:39.660984+0000","last_undegraded":"2023-11-24T07:58:39.660984+0000","last_fullsized":"2023-11-24T07:58:39.660984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670254+0000","last_change":"2023-11-24T07:58:38.667006+0000","last_active":"2023-11-24T07:58:39.670254+0000","last_peered":"2023-11-24T07:58:39.670254+0000","last_clean":"2023-11-24T07:58:39.670254+0000","last_became_active":"2023-11-24T07:58:38.665254+0000","last_became_peered":"2023-11-24T07:58:38.665254+0000","last_unstale":"2023-11-24T07:58:39.670254+0000","last_undegraded":"2023-11-24T07:58:39.670254+0000","last_fullsized":"2023-11-24T07:58:39.670254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655425+0000","last_change":"2023-11-24T07:58:37.660588+0000","last_active":"2023-11-24T07:58:39.655425+0000","last_peered":"2023-11-24T07:58:39.655425+0000","last_clean":"2023-11-24T07:58:39.655425+0000","last_became_active":"2023-11-24T07:58:37.660295+0000","last_became_peered":"2023-11-24T07:58:37.660295+0000","last_unstale":"2023-11-24T07:58:39.655425+0000","last_undegraded":"2023-11-24T07:58:39.655425+0000","last_fullsized":"2023-11-24T07:58:39.655425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655528+0000","last_change":"2023-11-24T07:58:38.671305+0000","last_active":"2023-11-24T07:58:39.655528+0000","last_peered":"2023-11-24T07:58:39.655528+0000","last_clean":"2023-11-24T07:58:39.655528+0000","last_became_active":"2023-11-24T07:58:38.670974+0000","last_became_peered":"2023-11-24T07:58:38.670974+0000","last_unstale":"2023-11-24T07:58:39.655528+0000","last_undegraded":"2023-11-24T07:58:39.655528+0000","last_fullsized":"2023-11-24T07:58:39.655528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661837+0000","last_change":"2023-11-24T07:58:37.658192+0000","last_active":"2023-11-24T07:58:39.661837+0000","last_peered":"2023-11-24T07:58:39.661837+0000","last_clean":"2023-11-24T07:58:39.661837+0000","last_became_active":"2023-11-24T07:58:37.657283+0000","last_became_peered":"2023-11-24T07:58:37.657283+0000","last_unstale":"2023-11-24T07:58:39.661837+0000","last_undegraded":"2023-11-24T07:58:39.661837+0000","last_fullsized":"2023-11-24T07:58:39.661837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209719+0000","last_change":"2023-11-24T07:58:38.655330+0000","last_active":"2023-11-24T07:58:40.209719+0000","last_peered":"2023-11-24T07:58:40.209719+0000","last_clean":"2023-11-24T07:58:40.209719+0000","last_became_active":"2023-11-24T07:58:38.654988+0000","last_became_peered":"2023-11-24T07:58:38.654988+0000","last_unstale":"2023-11-24T07:58:40.209719+0000","last_undegraded":"2023-11-24T07:58:40.209719+0000","last_fullsized":"2023-11-24T07:58:40.209719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661550+0000","last_change":"2023-11-24T07:58:37.663203+0000","last_active":"2023-11-24T07:58:39.661550+0000","last_peered":"2023-11-24T07:58:39.661550+0000","last_clean":"2023-11-24T07:58:39.661550+0000","last_became_active":"2023-11-24T07:58:37.662842+0000","last_became_peered":"2023-11-24T07:58:37.662842+0000","last_unstale":"2023-11-24T07:58:39.661550+0000","last_undegraded":"2023-11-24T07:58:39.661550+0000","last_fullsized":"2023-11-24T07:58:39.661550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662636+0000","last_change":"2023-11-24T07:58:38.662184+0000","last_active":"2023-11-24T07:58:39.662636+0000","last_peered":"2023-11-24T07:58:39.662636+0000","last_clean":"2023-11-24T07:58:39.662636+0000","last_became_active":"2023-11-24T07:58:38.661870+0000","last_became_peered":"2023-11-24T07:58:38.661870+0000","last_unstale":"2023-11-24T07:58:39.662636+0000","last_undegraded":"2023-11-24T07:58:39.662636+0000","last_fullsized":"2023-11-24T07:58:39.662636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662519+0000","last_change":"2023-11-24T07:58:37.663425+0000","last_active":"2023-11-24T07:58:39.662519+0000","last_peered":"2023-11-24T07:58:39.662519+0000","last_clean":"2023-11-24T07:58:39.662519+0000","last_became_active":"2023-11-24T07:58:37.663165+0000","last_became_peered":"2023-11-24T07:58:37.663165+0000","last_unstale":"2023-11-24T07:58:39.662519+0000","last_undegraded":"2023-11-24T07:58:39.662519+0000","last_fullsized":"2023-11-24T07:58:39.662519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670128+0000","last_change":"2023-11-24T07:58:38.669831+0000","last_active":"2023-11-24T07:58:39.670128+0000","last_peered":"2023-11-24T07:58:39.670128+0000","last_clean":"2023-11-24T07:58:39.670128+0000","last_became_active":"2023-11-24T07:58:38.669524+0000","last_became_peered":"2023-11-24T07:58:38.669524+0000","last_unstale":"2023-11-24T07:58:39.670128+0000","last_undegraded":"2023-11-24T07:58:39.670128+0000","last_fullsized":"2023-11-24T07:58:39.670128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209893+0000","last_change":"2023-11-24T07:58:37.656248+0000","last_active":"2023-11-24T07:58:40.209893+0000","last_peered":"2023-11-24T07:58:40.209893+0000","last_clean":"2023-11-24T07:58:40.209893+0000","last_became_active":"2023-11-24T07:58:37.655538+0000","last_became_peered":"2023-11-24T07:58:37.655538+0000","last_unstale":"2023-11-24T07:58:40.209893+0000","last_undegraded":"2023-11-24T07:58:40.209893+0000","last_fullsized":"2023-11-24T07:58:40.209893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210389+0000","last_change":"2023-11-24T07:58:38.660386+0000","last_active":"2023-11-24T07:58:40.210389+0000","last_peered":"2023-11-24T07:58:40.210389+0000","last_clean":"2023-11-24T07:58:40.210389+0000","last_became_active":"2023-11-24T07:58:38.659454+0000","last_became_peered":"2023-11-24T07:58:38.659454+0000","last_unstale":"2023-11-24T07:58:40.210389+0000","last_undegraded":"2023-11-24T07:58:40.210389+0000","last_fullsized":"2023-11-24T07:58:40.210389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661142+0000","last_change":"2023-11-24T07:58:37.663115+0000","last_active":"2023-11-24T07:58:39.661142+0000","last_peered":"2023-11-24T07:58:39.661142+0000","last_clean":"2023-11-24T07:58:39.661142+0000","last_became_active":"2023-11-24T07:58:37.662759+0000","last_became_peered":"2023-11-24T07:58:37.662759+0000","last_unstale":"2023-11-24T07:58:39.661142+0000","last_undegraded":"2023-11-24T07:58:39.661142+0000","last_fullsized":"2023-11-24T07:58:39.661142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210514+0000","last_change":"2023-11-24T07:58:38.677184+0000","last_active":"2023-11-24T07:58:40.210514+0000","last_peered":"2023-11-24T07:58:40.210514+0000","last_clean":"2023-11-24T07:58:40.210514+0000","last_became_active":"2023-11-24T07:58:38.676729+0000","last_became_peered":"2023-11-24T07:58:38.676729+0000","last_unstale":"2023-11-24T07:58:40.210514+0000","last_undegraded":"2023-11-24T07:58:40.210514+0000","last_fullsized":"2023-11-24T07:58:40.210514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670164+0000","last_change":"2023-11-24T07:58:37.659853+0000","last_active":"2023-11-24T07:58:39.670164+0000","last_peered":"2023-11-24T07:58:39.670164+0000","last_clean":"2023-11-24T07:58:39.670164+0000","last_became_active":"2023-11-24T07:58:37.659563+0000","last_became_peered":"2023-11-24T07:58:37.659563+0000","last_unstale":"2023-11-24T07:58:39.670164+0000","last_undegraded":"2023-11-24T07:58:39.670164+0000","last_fullsized":"2023-11-24T07:58:39.670164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670259+0000","last_change":"2023-11-24T07:58:38.668668+0000","last_active":"2023-11-24T07:58:39.670259+0000","last_peered":"2023-11-24T07:58:39.670259+0000","last_clean":"2023-11-24T07:58:39.670259+0000","last_became_active":"2023-11-24T07:58:38.668181+0000","last_became_peered":"2023-11-24T07:58:38.668181+0000","last_unstale":"2023-11-24T07:58:39.670259+0000","last_undegraded":"2023-11-24T07:58:39.670259+0000","last_fullsized":"2023-11-24T07:58:39.670259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662643+0000","last_change":"2023-11-24T07:58:37.656567+0000","last_active":"2023-11-24T07:58:39.662643+0000","last_peered":"2023-11-24T07:58:39.662643+0000","last_clean":"2023-11-24T07:58:39.662643+0000","last_became_active":"2023-11-24T07:58:37.656024+0000","last_became_peered":"2023-11-24T07:58:37.656024+0000","last_unstale":"2023-11-24T07:58:39.662643+0000","last_undegraded":"2023-11-24T07:58:39.662643+0000","last_fullsized":"2023-11-24T07:58:39.662643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210630+0000","last_change":"2023-11-24T07:58:38.664336+0000","last_active":"2023-11-24T07:58:40.210630+0000","last_peered":"2023-11-24T07:58:40.210630+0000","last_clean":"2023-11-24T07:58:40.210630+0000","last_became_active":"2023-11-24T07:58:38.664080+0000","last_became_peered":"2023-11-24T07:58:38.664080+0000","last_unstale":"2023-11-24T07:58:40.210630+0000","last_undegraded":"2023-11-24T07:58:40.210630+0000","last_fullsized":"2023-11-24T07:58:40.210630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660834+0000","last_change":"2023-11-24T07:58:37.660791+0000","last_active":"2023-11-24T07:58:39.660834+0000","last_peered":"2023-11-24T07:58:39.660834+0000","last_clean":"2023-11-24T07:58:39.660834+0000","last_became_active":"2023-11-24T07:58:37.660488+0000","last_became_peered":"2023-11-24T07:58:37.660488+0000","last_unstale":"2023-11-24T07:58:39.660834+0000","last_undegraded":"2023-11-24T07:58:39.660834+0000","last_fullsized":"2023-11-24T07:58:39.660834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660799+0000","last_change":"2023-11-24T07:58:38.670298+0000","last_active":"2023-11-24T07:58:39.660799+0000","last_peered":"2023-11-24T07:58:39.660799+0000","last_clean":"2023-11-24T07:58:39.660799+0000","last_became_active":"2023-11-24T07:58:38.669956+0000","last_became_peered":"2023-11-24T07:58:38.669956+0000","last_unstale":"2023-11-24T07:58:39.660799+0000","last_undegraded":"2023-11-24T07:58:39.660799+0000","last_fullsized":"2023-11-24T07:58:39.660799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209457+0000","last_change":"2023-11-24T07:58:37.659643+0000","last_active":"2023-11-24T07:58:40.209457+0000","last_peered":"2023-11-24T07:58:40.209457+0000","last_clean":"2023-11-24T07:58:40.209457+0000","last_became_active":"2023-11-24T07:58:37.659317+0000","last_became_peered":"2023-11-24T07:58:37.659317+0000","last_unstale":"2023-11-24T07:58:40.209457+0000","last_undegraded":"2023-11-24T07:58:40.209457+0000","last_fullsized":"2023-11-24T07:58:40.209457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:32.387984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209982+0000","last_change":"2023-11-24T07:58:38.667271+0000","last_active":"2023-11-24T07:58:40.209982+0000","last_peered":"2023-11-24T07:58:40.209982+0000","last_clean":"2023-11-24T07:58:40.209982+0000","last_became_active":"2023-11-24T07:58:38.666904+0000","last_became_peered":"2023-11-24T07:58:38.666904+0000","last_unstale":"2023-11-24T07:58:40.209982+0000","last_undegraded":"2023-11-24T07:58:40.209982+0000","last_fullsized":"2023-11-24T07:58:40.209982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210573+0000","last_change":"2023-11-24T07:58:37.657085+0000","last_active":"2023-11-24T07:58:40.210573+0000","last_peered":"2023-11-24T07:58:40.210573+0000","last_clean":"2023-11-24T07:58:40.210573+0000","last_became_active":"2023-11-24T07:58:37.656676+0000","last_became_peered":"2023-11-24T07:58:37.656676+0000","last_unstale":"2023-11-24T07:58:40.210573+0000","last_undegraded":"2023-11-24T07:58:40.210573+0000","last_fullsized":"2023-11-24T07:58:40.210573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:45.701317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210306+0000","last_change":"2023-11-24T07:58:38.664417+0000","last_active":"2023-11-24T07:58:40.210306+0000","last_peered":"2023-11-24T07:58:40.210306+0000","last_clean":"2023-11-24T07:58:40.210306+0000","last_became_active":"2023-11-24T07:58:38.664161+0000","last_became_peered":"2023-11-24T07:58:38.664161+0000","last_unstale":"2023-11-24T07:58:40.210306+0000","last_undegraded":"2023-11-24T07:58:40.210306+0000","last_fullsized":"2023-11-24T07:58:40.210306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209807+0000","last_change":"2023-11-24T07:58:37.660866+0000","last_active":"2023-11-24T07:58:40.209807+0000","last_peered":"2023-11-24T07:58:40.209807+0000","last_clean":"2023-11-24T07:58:40.209807+0000","last_became_active":"2023-11-24T07:58:37.660577+0000","last_became_peered":"2023-11-24T07:58:37.660577+0000","last_unstale":"2023-11-24T07:58:40.209807+0000","last_undegraded":"2023-11-24T07:58:40.209807+0000","last_fullsized":"2023-11-24T07:58:40.209807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662441+0000","last_change":"2023-11-24T07:58:38.674685+0000","last_active":"2023-11-24T07:58:39.662441+0000","last_peered":"2023-11-24T07:58:39.662441+0000","last_clean":"2023-11-24T07:58:39.662441+0000","last_became_active":"2023-11-24T07:58:38.674366+0000","last_became_peered":"2023-11-24T07:58:38.674366+0000","last_unstale":"2023-11-24T07:58:39.662441+0000","last_undegraded":"2023-11-24T07:58:39.662441+0000","last_fullsized":"2023-11-24T07:58:39.662441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662580+0000","last_change":"2023-11-24T07:58:37.655735+0000","last_active":"2023-11-24T07:58:39.662580+0000","last_peered":"2023-11-24T07:58:39.662580+0000","last_clean":"2023-11-24T07:58:39.662580+0000","last_became_active":"2023-11-24T07:58:37.655444+0000","last_became_peered":"2023-11-24T07:58:37.655444+0000","last_unstale":"2023-11-24T07:58:39.662580+0000","last_undegraded":"2023-11-24T07:58:39.662580+0000","last_fullsized":"2023-11-24T07:58:39.662580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:10.569901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210250+0000","last_change":"2023-11-24T07:58:38.667440+0000","last_active":"2023-11-24T07:58:40.210250+0000","last_peered":"2023-11-24T07:58:40.210250+0000","last_clean":"2023-11-24T07:58:40.210250+0000","last_became_active":"2023-11-24T07:58:38.666914+0000","last_became_peered":"2023-11-24T07:58:38.666914+0000","last_unstale":"2023-11-24T07:58:40.210250+0000","last_undegraded":"2023-11-24T07:58:40.210250+0000","last_fullsized":"2023-11-24T07:58:40.210250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:19.761593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209468+0000","last_change":"2023-11-24T07:58:38.655018+0000","last_active":"2023-11-24T07:58:40.209468+0000","last_peered":"2023-11-24T07:58:40.209468+0000","last_clean":"2023-11-24T07:58:40.209468+0000","last_became_active":"2023-11-24T07:58:38.654604+0000","last_became_peered":"2023-11-24T07:58:38.654604+0000","last_unstale":"2023-11-24T07:58:40.209468+0000","last_undegraded":"2023-11-24T07:58:40.209468+0000","last_fullsized":"2023-11-24T07:58:40.209468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209541+0000","last_change":"2023-11-24T07:58:37.660113+0000","last_active":"2023-11-24T07:58:40.209541+0000","last_peered":"2023-11-24T07:58:40.209541+0000","last_clean":"2023-11-24T07:58:40.209541+0000","last_became_active":"2023-11-24T07:58:37.659747+0000","last_became_peered":"2023-11-24T07:58:37.659747+0000","last_unstale":"2023-11-24T07:58:40.209541+0000","last_undegraded":"2023-11-24T07:58:40.209541+0000","last_fullsized":"2023-11-24T07:58:40.209541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:02.242362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655662+0000","last_change":"2023-11-24T07:58:38.670798+0000","last_active":"2023-11-24T07:58:39.655662+0000","last_peered":"2023-11-24T07:58:39.655662+0000","last_clean":"2023-11-24T07:58:39.655662+0000","last_became_active":"2023-11-24T07:58:38.670464+0000","last_became_peered":"2023-11-24T07:58:38.670464+0000","last_unstale":"2023-11-24T07:58:39.655662+0000","last_undegraded":"2023-11-24T07:58:39.655662+0000","last_fullsized":"2023-11-24T07:58:39.655662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662521+0000","last_change":"2023-11-24T07:58:37.661568+0000","last_active":"2023-11-24T07:58:39.662521+0000","last_peered":"2023-11-24T07:58:39.662521+0000","last_clean":"2023-11-24T07:58:39.662521+0000","last_became_active":"2023-11-24T07:58:37.661211+0000","last_became_peered":"2023-11-24T07:58:37.661211+0000","last_unstale":"2023-11-24T07:58:39.662521+0000","last_undegraded":"2023-11-24T07:58:39.662521+0000","last_fullsized":"2023-11-24T07:58:39.662521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:58:38.650810+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:58:38.650810+0000","last_peered":"2023-11-24T07:58:38.650810+0000","last_clean":"2023-11-24T07:58:38.650810+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:58:38.650810+0000","last_undegraded":"2023-11-24T07:58:38.650810+0000","last_fullsized":"2023-11-24T07:58:38.650810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669820+0000","last_change":"2023-11-24T07:58:38.667012+0000","last_active":"2023-11-24T07:58:39.669820+0000","last_peered":"2023-11-24T07:58:39.669820+0000","last_clean":"2023-11-24T07:58:39.669820+0000","last_became_active":"2023-11-24T07:58:38.666604+0000","last_became_peered":"2023-11-24T07:58:38.666604+0000","last_unstale":"2023-11-24T07:58:39.669820+0000","last_undegraded":"2023-11-24T07:58:39.669820+0000","last_fullsized":"2023-11-24T07:58:39.669820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.085407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209908+0000","last_change":"2023-11-24T07:58:37.655598+0000","last_active":"2023-11-24T07:58:40.209908+0000","last_peered":"2023-11-24T07:58:40.209908+0000","last_clean":"2023-11-24T07:58:40.209908+0000","last_became_active":"2023-11-24T07:58:37.655321+0000","last_became_peered":"2023-11-24T07:58:37.655321+0000","last_unstale":"2023-11-24T07:58:40.209908+0000","last_undegraded":"2023-11-24T07:58:40.209908+0000","last_fullsized":"2023-11-24T07:58:40.209908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210609+0000","last_change":"2023-11-24T07:58:37.659615+0000","last_active":"2023-11-24T07:58:40.210609+0000","last_peered":"2023-11-24T07:58:40.210609+0000","last_clean":"2023-11-24T07:58:40.210609+0000","last_became_active":"2023-11-24T07:58:37.659362+0000","last_became_peered":"2023-11-24T07:58:37.659362+0000","last_unstale":"2023-11-24T07:58:40.210609+0000","last_undegraded":"2023-11-24T07:58:40.210609+0000","last_fullsized":"2023-11-24T07:58:40.210609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209722+0000","last_change":"2023-11-24T07:58:38.655615+0000","last_active":"2023-11-24T07:58:40.209722+0000","last_peered":"2023-11-24T07:58:40.209722+0000","last_clean":"2023-11-24T07:58:40.209722+0000","last_became_active":"2023-11-24T07:58:38.654516+0000","last_became_peered":"2023-11-24T07:58:38.654516+0000","last_unstale":"2023-11-24T07:58:40.209722+0000","last_undegraded":"2023-11-24T07:58:40.209722+0000","last_fullsized":"2023-11-24T07:58:40.209722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.583888+0000","last_change":"2023-11-24T07:58:37.663218+0000","last_active":"2023-11-24T07:58:41.583888+0000","last_peered":"2023-11-24T07:58:41.583888+0000","last_clean":"2023-11-24T07:58:41.583888+0000","last_became_active":"2023-11-24T07:58:37.662867+0000","last_became_peered":"2023-11-24T07:58:37.662867+0000","last_unstale":"2023-11-24T07:58:41.583888+0000","last_undegraded":"2023-11-24T07:58:41.583888+0000","last_fullsized":"2023-11-24T07:58:41.583888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656341+0000","last_change":"2023-11-24T07:58:38.670569+0000","last_active":"2023-11-24T07:58:39.656341+0000","last_peered":"2023-11-24T07:58:39.656341+0000","last_clean":"2023-11-24T07:58:39.656341+0000","last_became_active":"2023-11-24T07:58:38.670239+0000","last_became_peered":"2023-11-24T07:58:38.670239+0000","last_unstale":"2023-11-24T07:58:39.656341+0000","last_undegraded":"2023-11-24T07:58:39.656341+0000","last_fullsized":"2023-11-24T07:58:39.656341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661516+0000","last_change":"2023-11-24T07:58:37.667212+0000","last_active":"2023-11-24T07:58:39.661516+0000","last_peered":"2023-11-24T07:58:39.661516+0000","last_clean":"2023-11-24T07:58:39.661516+0000","last_became_active":"2023-11-24T07:58:37.666944+0000","last_became_peered":"2023-11-24T07:58:37.666944+0000","last_unstale":"2023-11-24T07:58:39.661516+0000","last_undegraded":"2023-11-24T07:58:39.661516+0000","last_fullsized":"2023-11-24T07:58:39.661516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670071+0000","last_change":"2023-11-24T07:58:38.660314+0000","last_active":"2023-11-24T07:58:39.670071+0000","last_peered":"2023-11-24T07:58:39.670071+0000","last_clean":"2023-11-24T07:58:39.670071+0000","last_became_active":"2023-11-24T07:58:38.660005+0000","last_became_peered":"2023-11-24T07:58:38.660005+0000","last_unstale":"2023-11-24T07:58:39.670071+0000","last_undegraded":"2023-11-24T07:58:39.670071+0000","last_fullsized":"2023-11-24T07:58:39.670071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209750+0000","last_change":"2023-11-24T07:58:37.667045+0000","last_active":"2023-11-24T07:58:40.209750+0000","last_peered":"2023-11-24T07:58:40.209750+0000","last_clean":"2023-11-24T07:58:40.209750+0000","last_became_active":"2023-11-24T07:58:37.666766+0000","last_became_peered":"2023-11-24T07:58:37.666766+0000","last_unstale":"2023-11-24T07:58:40.209750+0000","last_undegraded":"2023-11-24T07:58:40.209750+0000","last_fullsized":"2023-11-24T07:58:40.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209814+0000","last_change":"2023-11-24T07:58:38.652046+0000","last_active":"2023-11-24T07:58:40.209814+0000","last_peered":"2023-11-24T07:58:40.209814+0000","last_clean":"2023-11-24T07:58:40.209814+0000","last_became_active":"2023-11-24T07:58:38.651762+0000","last_became_peered":"2023-11-24T07:58:38.651762+0000","last_unstale":"2023-11-24T07:58:40.209814+0000","last_undegraded":"2023-11-24T07:58:40.209814+0000","last_fullsized":"2023-11-24T07:58:40.209814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:28.237034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209556+0000","last_change":"2023-11-24T07:58:37.655838+0000","last_active":"2023-11-24T07:58:40.209556+0000","last_peered":"2023-11-24T07:58:40.209556+0000","last_clean":"2023-11-24T07:58:40.209556+0000","last_became_active":"2023-11-24T07:58:37.655553+0000","last_became_peered":"2023-11-24T07:58:37.655553+0000","last_unstale":"2023-11-24T07:58:40.209556+0000","last_undegraded":"2023-11-24T07:58:40.209556+0000","last_fullsized":"2023-11-24T07:58:40.209556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655413+0000","last_change":"2023-11-24T07:58:38.655539+0000","last_active":"2023-11-24T07:58:39.655413+0000","last_peered":"2023-11-24T07:58:39.655413+0000","last_clean":"2023-11-24T07:58:39.655413+0000","last_became_active":"2023-11-24T07:58:38.655169+0000","last_became_peered":"2023-11-24T07:58:38.655169+0000","last_unstale":"2023-11-24T07:58:39.655413+0000","last_undegraded":"2023-11-24T07:58:39.655413+0000","last_fullsized":"2023-11-24T07:58:39.655413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209712+0000","last_change":"2023-11-24T07:58:37.655841+0000","last_active":"2023-11-24T07:58:40.209712+0000","last_peered":"2023-11-24T07:58:40.209712+0000","last_clean":"2023-11-24T07:58:40.209712+0000","last_became_active":"2023-11-24T07:58:37.655487+0000","last_became_peered":"2023-11-24T07:58:37.655487+0000","last_unstale":"2023-11-24T07:58:40.209712+0000","last_undegraded":"2023-11-24T07:58:40.209712+0000","last_fullsized":"2023-11-24T07:58:40.209712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655699+0000","last_change":"2023-11-24T07:58:38.671051+0000","last_active":"2023-11-24T07:58:39.655699+0000","last_peered":"2023-11-24T07:58:39.655699+0000","last_clean":"2023-11-24T07:58:39.655699+0000","last_became_active":"2023-11-24T07:58:38.670571+0000","last_became_peered":"2023-11-24T07:58:38.670571+0000","last_unstale":"2023-11-24T07:58:39.655699+0000","last_undegraded":"2023-11-24T07:58:39.655699+0000","last_fullsized":"2023-11-24T07:58:39.655699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210427+0000","last_change":"2023-11-24T07:58:37.655295+0000","last_active":"2023-11-24T07:58:40.210427+0000","last_peered":"2023-11-24T07:58:40.210427+0000","last_clean":"2023-11-24T07:58:40.210427+0000","last_became_active":"2023-11-24T07:58:37.654767+0000","last_became_peered":"2023-11-24T07:58:37.654767+0000","last_unstale":"2023-11-24T07:58:40.210427+0000","last_undegraded":"2023-11-24T07:58:40.210427+0000","last_fullsized":"2023-11-24T07:58:40.210427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660877+0000","last_change":"2023-11-24T07:58:38.662792+0000","last_active":"2023-11-24T07:58:39.660877+0000","last_peered":"2023-11-24T07:58:39.660877+0000","last_clean":"2023-11-24T07:58:39.660877+0000","last_became_active":"2023-11-24T07:58:38.662492+0000","last_became_peered":"2023-11-24T07:58:38.662492+0000","last_unstale":"2023-11-24T07:58:39.660877+0000","last_undegraded":"2023-11-24T07:58:39.660877+0000","last_fullsized":"2023-11-24T07:58:39.660877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210083+0000","last_change":"2023-11-24T07:58:37.660634+0000","last_active":"2023-11-24T07:58:40.210083+0000","last_peered":"2023-11-24T07:58:40.210083+0000","last_clean":"2023-11-24T07:58:40.210083+0000","last_became_active":"2023-11-24T07:58:37.660340+0000","last_became_peered":"2023-11-24T07:58:37.660340+0000","last_unstale":"2023-11-24T07:58:40.210083+0000","last_undegraded":"2023-11-24T07:58:40.210083+0000","last_fullsized":"2023-11-24T07:58:40.210083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:04.555457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210612+0000","last_change":"2023-11-24T07:58:38.672774+0000","last_active":"2023-11-24T07:58:40.210612+0000","last_peered":"2023-11-24T07:58:40.210612+0000","last_clean":"2023-11-24T07:58:40.210612+0000","last_became_active":"2023-11-24T07:58:38.672498+0000","last_became_peered":"2023-11-24T07:58:38.672498+0000","last_unstale":"2023-11-24T07:58:40.210612+0000","last_undegraded":"2023-11-24T07:58:40.210612+0000","last_fullsized":"2023-11-24T07:58:40.210612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209839+0000","last_change":"2023-11-24T07:58:37.653547+0000","last_active":"2023-11-24T07:58:40.209839+0000","last_peered":"2023-11-24T07:58:40.209839+0000","last_clean":"2023-11-24T07:58:40.209839+0000","last_became_active":"2023-11-24T07:58:37.653148+0000","last_became_peered":"2023-11-24T07:58:37.653148+0000","last_unstale":"2023-11-24T07:58:40.209839+0000","last_undegraded":"2023-11-24T07:58:40.209839+0000","last_fullsized":"2023-11-24T07:58:40.209839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662278+0000","last_change":"2023-11-24T07:58:38.676919+0000","last_active":"2023-11-24T07:58:39.662278+0000","last_peered":"2023-11-24T07:58:39.662278+0000","last_clean":"2023-11-24T07:58:39.662278+0000","last_became_active":"2023-11-24T07:58:38.676622+0000","last_became_peered":"2023-11-24T07:58:38.676622+0000","last_unstale":"2023-11-24T07:58:39.662278+0000","last_undegraded":"2023-11-24T07:58:39.662278+0000","last_fullsized":"2023-11-24T07:58:39.662278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210143+0000","last_change":"2023-11-24T07:58:37.659572+0000","last_active":"2023-11-24T07:58:40.210143+0000","last_peered":"2023-11-24T07:58:40.210143+0000","last_clean":"2023-11-24T07:58:40.210143+0000","last_became_active":"2023-11-24T07:58:37.659273+0000","last_became_peered":"2023-11-24T07:58:37.659273+0000","last_unstale":"2023-11-24T07:58:40.210143+0000","last_undegraded":"2023-11-24T07:58:40.210143+0000","last_fullsized":"2023-11-24T07:58:40.210143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210569+0000","last_change":"2023-11-24T07:58:38.673691+0000","last_active":"2023-11-24T07:58:40.210569+0000","last_peered":"2023-11-24T07:58:40.210569+0000","last_clean":"2023-11-24T07:58:40.210569+0000","last_became_active":"2023-11-24T07:58:38.673092+0000","last_became_peered":"2023-11-24T07:58:38.673092+0000","last_unstale":"2023-11-24T07:58:40.210569+0000","last_undegraded":"2023-11-24T07:58:40.210569+0000","last_fullsized":"2023-11-24T07:58:40.210569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661147+0000","last_change":"2023-11-24T07:58:38.670271+0000","last_active":"2023-11-24T07:58:39.661147+0000","last_peered":"2023-11-24T07:58:39.661147+0000","last_clean":"2023-11-24T07:58:39.661147+0000","last_became_active":"2023-11-24T07:58:38.669965+0000","last_became_peered":"2023-11-24T07:58:38.669965+0000","last_unstale":"2023-11-24T07:58:39.661147+0000","last_undegraded":"2023-11-24T07:58:39.661147+0000","last_fullsized":"2023-11-24T07:58:39.661147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209683+0000","last_change":"2023-11-24T07:58:38.663275+0000","last_active":"2023-11-24T07:58:40.209683+0000","last_peered":"2023-11-24T07:58:40.209683+0000","last_clean":"2023-11-24T07:58:40.209683+0000","last_became_active":"2023-11-24T07:58:38.662928+0000","last_became_peered":"2023-11-24T07:58:38.662928+0000","last_unstale":"2023-11-24T07:58:40.209683+0000","last_undegraded":"2023-11-24T07:58:40.209683+0000","last_fullsized":"2023-11-24T07:58:40.209683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662492+0000","last_change":"2023-11-24T07:58:38.674085+0000","last_active":"2023-11-24T07:58:39.662492+0000","last_peered":"2023-11-24T07:58:39.662492+0000","last_clean":"2023-11-24T07:58:39.662492+0000","last_became_active":"2023-11-24T07:58:38.673780+0000","last_became_peered":"2023-11-24T07:58:38.673780+0000","last_unstale":"2023-11-24T07:58:39.662492+0000","last_undegraded":"2023-11-24T07:58:39.662492+0000","last_fullsized":"2023-11-24T07:58:39.662492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670095+0000","last_change":"2023-11-24T07:58:38.676742+0000","last_active":"2023-11-24T07:58:39.670095+0000","last_peered":"2023-11-24T07:58:39.670095+0000","last_clean":"2023-11-24T07:58:39.670095+0000","last_became_active":"2023-11-24T07:58:38.676422+0000","last_became_peered":"2023-11-24T07:58:38.676422+0000","last_unstale":"2023-11-24T07:58:39.670095+0000","last_undegraded":"2023-11-24T07:58:39.670095+0000","last_fullsized":"2023-11-24T07:58:39.670095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655920+0000","last_change":"2023-11-24T07:58:38.674946+0000","last_active":"2023-11-24T07:58:39.655920+0000","last_peered":"2023-11-24T07:58:39.655920+0000","last_clean":"2023-11-24T07:58:39.655920+0000","last_became_active":"2023-11-24T07:58:38.674458+0000","last_became_peered":"2023-11-24T07:58:38.674458+0000","last_unstale":"2023-11-24T07:58:39.655920+0000","last_undegraded":"2023-11-24T07:58:39.655920+0000","last_fullsized":"2023-11-24T07:58:39.655920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209765+0000","last_change":"2023-11-24T07:58:38.656486+0000","last_active":"2023-11-24T07:58:40.209765+0000","last_peered":"2023-11-24T07:58:40.209765+0000","last_clean":"2023-11-24T07:58:40.209765+0000","last_became_active":"2023-11-24T07:58:38.654844+0000","last_became_peered":"2023-11-24T07:58:38.654844+0000","last_unstale":"2023-11-24T07:58:40.209765+0000","last_undegraded":"2023-11-24T07:58:40.209765+0000","last_fullsized":"2023-11-24T07:58:40.209765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661231+0000","last_change":"2023-11-24T07:58:38.666517+0000","last_active":"2023-11-24T07:58:39.661231+0000","last_peered":"2023-11-24T07:58:39.661231+0000","last_clean":"2023-11-24T07:58:39.661231+0000","last_became_active":"2023-11-24T07:58:38.665821+0000","last_became_peered":"2023-11-24T07:58:38.665821+0000","last_unstale":"2023-11-24T07:58:39.661231+0000","last_undegraded":"2023-11-24T07:58:39.661231+0000","last_fullsized":"2023-11-24T07:58:39.661231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656201+0000","last_change":"2023-11-24T07:58:38.672413+0000","last_active":"2023-11-24T07:58:39.656201+0000","last_peered":"2023-11-24T07:58:39.656201+0000","last_clean":"2023-11-24T07:58:39.656201+0000","last_became_active":"2023-11-24T07:58:38.672131+0000","last_became_peered":"2023-11-24T07:58:38.672131+0000","last_unstale":"2023-11-24T07:58:39.656201+0000","last_undegraded":"2023-11-24T07:58:39.656201+0000","last_fullsized":"2023-11-24T07:58:39.656201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662001+0000","last_change":"2023-11-24T07:58:38.672368+0000","last_active":"2023-11-24T07:58:39.662001+0000","last_peered":"2023-11-24T07:58:39.662001+0000","last_clean":"2023-11-24T07:58:39.662001+0000","last_became_active":"2023-11-24T07:58:38.672038+0000","last_became_peered":"2023-11-24T07:58:38.672038+0000","last_unstale":"2023-11-24T07:58:39.662001+0000","last_undegraded":"2023-11-24T07:58:39.662001+0000","last_fullsized":"2023-11-24T07:58:39.662001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655743+0000","last_change":"2023-11-24T07:58:38.671825+0000","last_active":"2023-11-24T07:58:39.655743+0000","last_peered":"2023-11-24T07:58:39.655743+0000","last_clean":"2023-11-24T07:58:39.655743+0000","last_became_active":"2023-11-24T07:58:38.671530+0000","last_became_peered":"2023-11-24T07:58:38.671530+0000","last_unstale":"2023-11-24T07:58:39.655743+0000","last_undegraded":"2023-11-24T07:58:39.655743+0000","last_fullsized":"2023-11-24T07:58:39.655743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662135+0000","last_change":"2023-11-24T07:58:38.663149+0000","last_active":"2023-11-24T07:58:39.662135+0000","last_peered":"2023-11-24T07:58:39.662135+0000","last_clean":"2023-11-24T07:58:39.662135+0000","last_became_active":"2023-11-24T07:58:38.662826+0000","last_became_peered":"2023-11-24T07:58:38.662826+0000","last_unstale":"2023-11-24T07:58:39.662135+0000","last_undegraded":"2023-11-24T07:58:39.662135+0000","last_fullsized":"2023-11-24T07:58:39.662135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670177+0000","last_change":"2023-11-24T07:58:38.673409+0000","last_active":"2023-11-24T07:58:39.670177+0000","last_peered":"2023-11-24T07:58:39.670177+0000","last_clean":"2023-11-24T07:58:39.670177+0000","last_became_active":"2023-11-24T07:58:38.673109+0000","last_became_peered":"2023-11-24T07:58:38.673109+0000","last_unstale":"2023-11-24T07:58:39.670177+0000","last_undegraded":"2023-11-24T07:58:39.670177+0000","last_fullsized":"2023-11-24T07:58:39.670177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661303+0000","last_change":"2023-11-24T07:58:38.666773+0000","last_active":"2023-11-24T07:58:39.661303+0000","last_peered":"2023-11-24T07:58:39.661303+0000","last_clean":"2023-11-24T07:58:39.661303+0000","last_became_active":"2023-11-24T07:58:38.666430+0000","last_became_peered":"2023-11-24T07:58:38.666430+0000","last_unstale":"2023-11-24T07:58:39.661303+0000","last_undegraded":"2023-11-24T07:58:39.661303+0000","last_fullsized":"2023-11-24T07:58:39.661303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:47.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.774+0000 7f74b1a5e700 1 Processor -- start 2023-11-24T07:58:47.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.774+0000 7f74b1a5e700 1 -- start start 2023-11-24T07:58:47.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74b1a5e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74ac15e1e0 0x7f74ac0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74b1a5e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74ac0d3a00 0x7f74ac0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74b1a5e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d0db0 0x7f74ac06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74b1a5e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac0d3150 con 0x7f74ac15e1e0 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74b1a5e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac0d1240 con 0x7f74ac0d3a00 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74b1a5e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15e5c0 con 0x7f74ac0d0db0 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74ac15e1e0 0x7f74ac0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74ac15e1e0 0x7f74ac0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56888/0 (socket says 172.21.15.32:56888) 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 -- 172.21.15.32:0/2944462763 learned_addr learned my addr 172.21.15.32:0/2944462763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:47.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aa7fc700 1 --2- 172.21.15.32:0/2944462763 >> v2:172.21.15.123:3300/0 conn(0x7f74ac0d3a00 0x7f74ac0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74ab7fe700 1 --2- 172.21.15.32:0/2944462763 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d0db0 0x7f74ac06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 -- 172.21.15.32:0/2944462763 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d0db0 msgr2=0x7f74ac06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 --2- 172.21.15.32:0/2944462763 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d0db0 0x7f74ac06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 -- 172.21.15.32:0/2944462763 >> v2:172.21.15.123:3300/0 conn(0x7f74ac0d3a00 msgr2=0x7f74ac0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 --2- 172.21.15.32:0/2944462763 >> v2:172.21.15.123:3300/0 conn(0x7f74ac0d3a00 0x7f74ac0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 -- 172.21.15.32:0/2944462763 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac0d20b0 con 0x7f74ac15e1e0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.775+0000 7f74aaffd700 1 --2- 172.21.15.32:0/2944462763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac15e1e0 0x7f74ac0d34c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f74ac161ba0 tx=0x7f749c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75837bcd405db2b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74a9ffb700 1 -- 172.21.15.32:0/2944462763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00b860 con 0x7f74ac15e1e0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74a9ffb700 1 -- 172.21.15.32:0/2944462763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c00b9c0 con 0x7f74ac15e1e0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74a9ffb700 1 -- 172.21.15.32:0/2944462763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00bd50 con 0x7f74ac15e1e0 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 -- 172.21.15.32:0/2944462763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac15e1e0 msgr2=0x7f74ac0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/2944462763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac15e1e0 0x7f74ac0d34c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f74ac161ba0 tx=0x7f749c009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 -- 172.21.15.32:0/2944462763 shutdown_connections 2023-11-24T07:58:47.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/2944462763 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d0db0 0x7f74ac06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/2944462763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac15e1e0 0x7f74ac0d34c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/2944462763 >> v2:172.21.15.123:3300/0 conn(0x7f74ac0d3a00 0x7f74ac0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 -- 172.21.15.32:0/2944462763 >> 172.21.15.32:0/2944462763 conn(0x7f74ac0b5b50 msgr2=0x7f74ac0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 -- 172.21.15.32:0/2944462763 shutdown_connections 2023-11-24T07:58:47.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.776+0000 7f74b1a5e700 1 -- 172.21.15.32:0/2944462763 wait complete. 2023-11-24T07:58:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.777+0000 7f74b1a5e700 1 Processor -- start 2023-11-24T07:58:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 -- start start 2023-11-24T07:58:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 0x7f74ac0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d3a00 0x7f74ac15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74ac15e1e0 0x7f74ac156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15b370 con 0x7f74ac0d0db0 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0cad00 con 0x7f74ac15e1e0 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74b1a5e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15adb0 con 0x7f74ac0d3a00 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 0x7f74ac0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 0x7f74ac0ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56892/0 (socket says 172.21.15.32:56892) 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.778+0000 7f74aaffd700 1 -- 172.21.15.32:0/4059746038 learned_addr learned my addr 172.21.15.32:0/4059746038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:47.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74ab7fe700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74ac15e1e0 0x7f74ac156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aa7fc700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d3a00 0x7f74ac15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aaffd700 1 -- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d3a00 msgr2=0x7f74ac15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aaffd700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d3a00 0x7f74ac15a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aaffd700 1 -- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74ac15e1e0 msgr2=0x7f74ac156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aaffd700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74ac15e1e0 0x7f74ac156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aaffd700 1 -- 172.21.15.32:0/4059746038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c009550 con 0x7f74ac0d0db0 2023-11-24T07:58:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74ab7fe700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74ac15e1e0 0x7f74ac156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74aaffd700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 0x7f74ac0ca7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f749c009130 tx=0x7f749c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0045d0 con 0x7f74ac0d0db0 2023-11-24T07:58:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c01e070 con 0x7f74ac0d0db0 2023-11-24T07:58:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac15b090 con 0x7f74ac0d0db0 2023-11-24T07:58:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0038a0 con 0x7f74ac0d0db0 2023-11-24T07:58:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.779+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac0cbda0 con 0x7f74ac0d0db0 2023-11-24T07:58:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.780+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac0cb920 con 0x7f74ac0d0db0 2023-11-24T07:58:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.780+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f749c003a00 con 0x7f74ac0d0db0 2023-11-24T07:58:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.781+0000 7f7493fff700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f749806aed0 0x7f749806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.781+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f749c020070 con 0x7f74ac0d0db0 2023-11-24T07:58:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.781+0000 7f74aa7fc700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f749806aed0 0x7f749806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.782+0000 7f74aa7fc700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f749806aed0 0x7f749806d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0d3e30 tx=0x7f74a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.785+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74ac0cb920 con 0x7f74ac0d0db0 2023-11-24T07:58:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.983+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74ac0d2950 con 0x7f749806aed0 2023-11-24T07:58:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.991+0000 7f7493fff700 1 -- 172.21.15.32:0/4059746038 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264533 (secure 0 0 0) 0x7f74a0015470 con 0x7f749806aed0 2023-11-24T07:58:47.995 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f749806aed0 msgr2=0x7f749806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f749806aed0 0x7f749806d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0d3e30 tx=0x7f74a0008040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 msgr2=0x7f74ac0ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 0x7f74ac0ca7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f749c009130 tx=0x7f749c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 shutdown_connections 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f749806aed0 0x7f749806d390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74ac15e1e0 0x7f74ac156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ac0d0db0 0x7f74ac0ca7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 --2- 172.21.15.32:0/4059746038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f74ac0d3a00 0x7f74ac15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 >> 172.21.15.32:0/4059746038 conn(0x7f74ac0b5b50 msgr2=0x7f74ac15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 shutdown_connections 2023-11-24T07:58:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:47.994+0000 7f74b1a5e700 1 -- 172.21.15.32:0/4059746038 wait complete. 2023-11-24T07:58:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T07:58:48.008 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T07:58:46.531391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219104,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755616,"statfs":{"total":773094113280,"available":772869750784,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210697+0000","last_change":"2023-11-24T07:58:38.673717+0000","last_active":"2023-11-24T07:58:40.210697+0000","last_peered":"2023-11-24T07:58:40.210697+0000","last_clean":"2023-11-24T07:58:40.210697+0000","last_became_active":"2023-11-24T07:58:38.673289+0000","last_became_peered":"2023-11-24T07:58:38.673289+0000","last_unstale":"2023-11-24T07:58:40.210697+0000","last_undegraded":"2023-11-24T07:58:40.210697+0000","last_fullsized":"2023-11-24T07:58:40.210697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:16:17.544102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209942+0000","last_change":"2023-11-24T07:58:38.666572+0000","last_active":"2023-11-24T07:58:40.209942+0000","last_peered":"2023-11-24T07:58:40.209942+0000","last_clean":"2023-11-24T07:58:40.209942+0000","last_became_active":"2023-11-24T07:58:38.666220+0000","last_became_peered":"2023-11-24T07:58:38.666220+0000","last_unstale":"2023-11-24T07:58:40.209942+0000","last_undegraded":"2023-11-24T07:58:40.209942+0000","last_fullsized":"2023-11-24T07:58:40.209942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670074+0000","last_change":"2023-11-24T07:58:38.664139+0000","last_active":"2023-11-24T07:58:39.670074+0000","last_peered":"2023-11-24T07:58:39.670074+0000","last_clean":"2023-11-24T07:58:39.670074+0000","last_became_active":"2023-11-24T07:58:38.660601+0000","last_became_peered":"2023-11-24T07:58:38.660601+0000","last_unstale":"2023-11-24T07:58:39.670074+0000","last_undegraded":"2023-11-24T07:58:39.670074+0000","last_fullsized":"2023-11-24T07:58:39.670074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662796+0000","last_change":"2023-11-24T07:58:38.670834+0000","last_active":"2023-11-24T07:58:39.662796+0000","last_peered":"2023-11-24T07:58:39.662796+0000","last_clean":"2023-11-24T07:58:39.662796+0000","last_became_active":"2023-11-24T07:58:38.670567+0000","last_became_peered":"2023-11-24T07:58:38.670567+0000","last_unstale":"2023-11-24T07:58:39.662796+0000","last_undegraded":"2023-11-24T07:58:39.662796+0000","last_fullsized":"2023-11-24T07:58:39.662796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210112+0000","last_change":"2023-11-24T07:58:38.657900+0000","last_active":"2023-11-24T07:58:40.210112+0000","last_peered":"2023-11-24T07:58:40.210112+0000","last_clean":"2023-11-24T07:58:40.210112+0000","last_became_active":"2023-11-24T07:58:38.657374+0000","last_became_peered":"2023-11-24T07:58:38.657374+0000","last_unstale":"2023-11-24T07:58:40.210112+0000","last_undegraded":"2023-11-24T07:58:40.210112+0000","last_fullsized":"2023-11-24T07:58:40.210112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662085+0000","last_change":"2023-11-24T07:58:38.668973+0000","last_active":"2023-11-24T07:58:39.662085+0000","last_peered":"2023-11-24T07:58:39.662085+0000","last_clean":"2023-11-24T07:58:39.662085+0000","last_became_active":"2023-11-24T07:58:38.668660+0000","last_became_peered":"2023-11-24T07:58:38.668660+0000","last_unstale":"2023-11-24T07:58:39.662085+0000","last_undegraded":"2023-11-24T07:58:39.662085+0000","last_fullsized":"2023-11-24T07:58:39.662085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209727+0000","last_change":"2023-11-24T07:58:38.670604+0000","last_active":"2023-11-24T07:58:40.209727+0000","last_peered":"2023-11-24T07:58:40.209727+0000","last_clean":"2023-11-24T07:58:40.209727+0000","last_became_active":"2023-11-24T07:58:38.670304+0000","last_became_peered":"2023-11-24T07:58:38.670304+0000","last_unstale":"2023-11-24T07:58:40.209727+0000","last_undegraded":"2023-11-24T07:58:40.209727+0000","last_fullsized":"2023-11-24T07:58:40.209727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655626+0000","last_change":"2023-11-24T07:58:38.671094+0000","last_active":"2023-11-24T07:58:39.655626+0000","last_peered":"2023-11-24T07:58:39.655626+0000","last_clean":"2023-11-24T07:58:39.655626+0000","last_became_active":"2023-11-24T07:58:38.670534+0000","last_became_peered":"2023-11-24T07:58:38.670534+0000","last_unstale":"2023-11-24T07:58:39.655626+0000","last_undegraded":"2023-11-24T07:58:39.655626+0000","last_fullsized":"2023-11-24T07:58:39.655626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210625+0000","last_change":"2023-11-24T07:58:38.673877+0000","last_active":"2023-11-24T07:58:40.210625+0000","last_peered":"2023-11-24T07:58:40.210625+0000","last_clean":"2023-11-24T07:58:40.210625+0000","last_became_active":"2023-11-24T07:58:38.673449+0000","last_became_peered":"2023-11-24T07:58:38.673449+0000","last_unstale":"2023-11-24T07:58:40.210625+0000","last_undegraded":"2023-11-24T07:58:40.210625+0000","last_fullsized":"2023-11-24T07:58:40.210625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:05.634735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661371+0000","last_change":"2023-11-24T07:58:38.667090+0000","last_active":"2023-11-24T07:58:39.661371+0000","last_peered":"2023-11-24T07:58:39.661371+0000","last_clean":"2023-11-24T07:58:39.661371+0000","last_became_active":"2023-11-24T07:58:38.665486+0000","last_became_peered":"2023-11-24T07:58:38.665486+0000","last_unstale":"2023-11-24T07:58:39.661371+0000","last_undegraded":"2023-11-24T07:58:39.661371+0000","last_fullsized":"2023-11-24T07:58:39.661371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661214+0000","last_change":"2023-11-24T07:58:38.666051+0000","last_active":"2023-11-24T07:58:39.661214+0000","last_peered":"2023-11-24T07:58:39.661214+0000","last_clean":"2023-11-24T07:58:39.661214+0000","last_became_active":"2023-11-24T07:58:38.665574+0000","last_became_peered":"2023-11-24T07:58:38.665574+0000","last_unstale":"2023-11-24T07:58:39.661214+0000","last_undegraded":"2023-11-24T07:58:39.661214+0000","last_fullsized":"2023-11-24T07:58:39.661214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210541+0000","last_change":"2023-11-24T07:58:38.667776+0000","last_active":"2023-11-24T07:58:40.210541+0000","last_peered":"2023-11-24T07:58:40.210541+0000","last_clean":"2023-11-24T07:58:40.210541+0000","last_became_active":"2023-11-24T07:58:38.667513+0000","last_became_peered":"2023-11-24T07:58:38.667513+0000","last_unstale":"2023-11-24T07:58:40.210541+0000","last_undegraded":"2023-11-24T07:58:40.210541+0000","last_fullsized":"2023-11-24T07:58:40.210541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655517+0000","last_change":"2023-11-24T07:58:38.672201+0000","last_active":"2023-11-24T07:58:39.655517+0000","last_peered":"2023-11-24T07:58:39.655517+0000","last_clean":"2023-11-24T07:58:39.655517+0000","last_became_active":"2023-11-24T07:58:38.671869+0000","last_became_peered":"2023-11-24T07:58:38.671869+0000","last_unstale":"2023-11-24T07:58:39.655517+0000","last_undegraded":"2023-11-24T07:58:39.655517+0000","last_fullsized":"2023-11-24T07:58:39.655517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670156+0000","last_change":"2023-11-24T07:58:38.662685+0000","last_active":"2023-11-24T07:58:39.670156+0000","last_peered":"2023-11-24T07:58:39.670156+0000","last_clean":"2023-11-24T07:58:39.670156+0000","last_became_active":"2023-11-24T07:58:38.660043+0000","last_became_peered":"2023-11-24T07:58:38.660043+0000","last_unstale":"2023-11-24T07:58:39.670156+0000","last_undegraded":"2023-11-24T07:58:39.670156+0000","last_fullsized":"2023-11-24T07:58:39.670156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209459+0000","last_change":"2023-11-24T07:58:38.667653+0000","last_active":"2023-11-24T07:58:40.209459+0000","last_peered":"2023-11-24T07:58:40.209459+0000","last_clean":"2023-11-24T07:58:40.209459+0000","last_became_active":"2023-11-24T07:58:38.667388+0000","last_became_peered":"2023-11-24T07:58:38.667388+0000","last_unstale":"2023-11-24T07:58:40.209459+0000","last_undegraded":"2023-11-24T07:58:40.209459+0000","last_fullsized":"2023-11-24T07:58:40.209459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661870+0000","last_change":"2023-11-24T07:58:38.669289+0000","last_active":"2023-11-24T07:58:39.661870+0000","last_peered":"2023-11-24T07:58:39.661870+0000","last_clean":"2023-11-24T07:58:39.661870+0000","last_became_active":"2023-11-24T07:58:38.668715+0000","last_became_peered":"2023-11-24T07:58:38.668715+0000","last_unstale":"2023-11-24T07:58:39.661870+0000","last_undegraded":"2023-11-24T07:58:39.661870+0000","last_fullsized":"2023-11-24T07:58:39.661870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662528+0000","last_change":"2023-11-24T07:58:38.674765+0000","last_active":"2023-11-24T07:58:39.662528+0000","last_peered":"2023-11-24T07:58:39.662528+0000","last_clean":"2023-11-24T07:58:39.662528+0000","last_became_active":"2023-11-24T07:58:38.674479+0000","last_became_peered":"2023-11-24T07:58:38.674479+0000","last_unstale":"2023-11-24T07:58:39.662528+0000","last_undegraded":"2023-11-24T07:58:39.662528+0000","last_fullsized":"2023-11-24T07:58:39.662528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210293+0000","last_change":"2023-11-24T07:58:38.667446+0000","last_active":"2023-11-24T07:58:40.210293+0000","last_peered":"2023-11-24T07:58:40.210293+0000","last_clean":"2023-11-24T07:58:40.210293+0000","last_became_active":"2023-11-24T07:58:38.667164+0000","last_became_peered":"2023-11-24T07:58:38.667164+0000","last_unstale":"2023-11-24T07:58:40.210293+0000","last_undegraded":"2023-11-24T07:58:40.210293+0000","last_fullsized":"2023-11-24T07:58:40.210293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:52.879745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210359+0000","last_change":"2023-11-24T07:58:38.663509+0000","last_active":"2023-11-24T07:58:40.210359+0000","last_peered":"2023-11-24T07:58:40.210359+0000","last_clean":"2023-11-24T07:58:40.210359+0000","last_became_active":"2023-11-24T07:58:38.661576+0000","last_became_peered":"2023-11-24T07:58:38.661576+0000","last_unstale":"2023-11-24T07:58:40.210359+0000","last_undegraded":"2023-11-24T07:58:40.210359+0000","last_fullsized":"2023-11-24T07:58:40.210359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669966+0000","last_change":"2023-11-24T07:58:37.664997+0000","last_active":"2023-11-24T07:58:39.669966+0000","last_peered":"2023-11-24T07:58:39.669966+0000","last_clean":"2023-11-24T07:58:39.669966+0000","last_became_active":"2023-11-24T07:58:37.664509+0000","last_became_peered":"2023-11-24T07:58:37.664509+0000","last_unstale":"2023-11-24T07:58:39.669966+0000","last_undegraded":"2023-11-24T07:58:39.669966+0000","last_fullsized":"2023-11-24T07:58:39.669966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656179+0000","last_change":"2023-11-24T07:58:38.671945+0000","last_active":"2023-11-24T07:58:39.656179+0000","last_peered":"2023-11-24T07:58:39.656179+0000","last_clean":"2023-11-24T07:58:39.656179+0000","last_became_active":"2023-11-24T07:58:38.671503+0000","last_became_peered":"2023-11-24T07:58:38.671503+0000","last_unstale":"2023-11-24T07:58:39.656179+0000","last_undegraded":"2023-11-24T07:58:39.656179+0000","last_fullsized":"2023-11-24T07:58:39.656179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.866271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656004+0000","last_change":"2023-11-24T07:58:37.664969+0000","last_active":"2023-11-24T07:58:39.656004+0000","last_peered":"2023-11-24T07:58:39.656004+0000","last_clean":"2023-11-24T07:58:39.656004+0000","last_became_active":"2023-11-24T07:58:37.664661+0000","last_became_peered":"2023-11-24T07:58:37.664661+0000","last_unstale":"2023-11-24T07:58:39.656004+0000","last_undegraded":"2023-11-24T07:58:39.656004+0000","last_fullsized":"2023-11-24T07:58:39.656004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670016+0000","last_change":"2023-11-24T07:58:38.670070+0000","last_active":"2023-11-24T07:58:39.670016+0000","last_peered":"2023-11-24T07:58:39.670016+0000","last_clean":"2023-11-24T07:58:39.670016+0000","last_became_active":"2023-11-24T07:58:38.669739+0000","last_became_peered":"2023-11-24T07:58:38.669739+0000","last_unstale":"2023-11-24T07:58:39.670016+0000","last_undegraded":"2023-11-24T07:58:39.670016+0000","last_fullsized":"2023-11-24T07:58:39.670016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:04.076457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210695+0000","last_change":"2023-11-24T07:58:37.663122+0000","last_active":"2023-11-24T07:58:40.210695+0000","last_peered":"2023-11-24T07:58:40.210695+0000","last_clean":"2023-11-24T07:58:40.210695+0000","last_became_active":"2023-11-24T07:58:37.662846+0000","last_became_peered":"2023-11-24T07:58:37.662846+0000","last_unstale":"2023-11-24T07:58:40.210695+0000","last_undegraded":"2023-11-24T07:58:40.210695+0000","last_fullsized":"2023-11-24T07:58:40.210695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:56.244472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662566+0000","last_change":"2023-11-24T07:58:38.669302+0000","last_active":"2023-11-24T07:58:39.662566+0000","last_peered":"2023-11-24T07:58:39.662566+0000","last_clean":"2023-11-24T07:58:39.662566+0000","last_became_active":"2023-11-24T07:58:38.669023+0000","last_became_peered":"2023-11-24T07:58:38.669023+0000","last_unstale":"2023-11-24T07:58:39.662566+0000","last_undegraded":"2023-11-24T07:58:39.662566+0000","last_fullsized":"2023-11-24T07:58:39.662566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210236+0000","last_change":"2023-11-24T07:58:37.659022+0000","last_active":"2023-11-24T07:58:40.210236+0000","last_peered":"2023-11-24T07:58:40.210236+0000","last_clean":"2023-11-24T07:58:40.210236+0000","last_became_active":"2023-11-24T07:58:37.658698+0000","last_became_peered":"2023-11-24T07:58:37.658698+0000","last_unstale":"2023-11-24T07:58:40.210236+0000","last_undegraded":"2023-11-24T07:58:40.210236+0000","last_fullsized":"2023-11-24T07:58:40.210236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662404+0000","last_change":"2023-11-24T07:58:38.669294+0000","last_active":"2023-11-24T07:58:39.662404+0000","last_peered":"2023-11-24T07:58:39.662404+0000","last_clean":"2023-11-24T07:58:39.662404+0000","last_became_active":"2023-11-24T07:58:38.668636+0000","last_became_peered":"2023-11-24T07:58:38.668636+0000","last_unstale":"2023-11-24T07:58:39.662404+0000","last_undegraded":"2023-11-24T07:58:39.662404+0000","last_fullsized":"2023-11-24T07:58:39.662404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655621+0000","last_change":"2023-11-24T07:58:37.655113+0000","last_active":"2023-11-24T07:58:39.655621+0000","last_peered":"2023-11-24T07:58:39.655621+0000","last_clean":"2023-11-24T07:58:39.655621+0000","last_became_active":"2023-11-24T07:58:37.654784+0000","last_became_peered":"2023-11-24T07:58:37.654784+0000","last_unstale":"2023-11-24T07:58:39.655621+0000","last_undegraded":"2023-11-24T07:58:39.655621+0000","last_fullsized":"2023-11-24T07:58:39.655621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209899+0000","last_change":"2023-11-24T07:58:38.653315+0000","last_active":"2023-11-24T07:58:40.209899+0000","last_peered":"2023-11-24T07:58:40.209899+0000","last_clean":"2023-11-24T07:58:40.209899+0000","last_became_active":"2023-11-24T07:58:38.653027+0000","last_became_peered":"2023-11-24T07:58:38.653027+0000","last_unstale":"2023-11-24T07:58:40.209899+0000","last_undegraded":"2023-11-24T07:58:40.209899+0000","last_fullsized":"2023-11-24T07:58:40.209899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:03.244592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660984+0000","last_change":"2023-11-24T07:58:37.660782+0000","last_active":"2023-11-24T07:58:39.660984+0000","last_peered":"2023-11-24T07:58:39.660984+0000","last_clean":"2023-11-24T07:58:39.660984+0000","last_became_active":"2023-11-24T07:58:37.660419+0000","last_became_peered":"2023-11-24T07:58:37.660419+0000","last_unstale":"2023-11-24T07:58:39.660984+0000","last_undegraded":"2023-11-24T07:58:39.660984+0000","last_fullsized":"2023-11-24T07:58:39.660984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670254+0000","last_change":"2023-11-24T07:58:38.667006+0000","last_active":"2023-11-24T07:58:39.670254+0000","last_peered":"2023-11-24T07:58:39.670254+0000","last_clean":"2023-11-24T07:58:39.670254+0000","last_became_active":"2023-11-24T07:58:38.665254+0000","last_became_peered":"2023-11-24T07:58:38.665254+0000","last_unstale":"2023-11-24T07:58:39.670254+0000","last_undegraded":"2023-11-24T07:58:39.670254+0000","last_fullsized":"2023-11-24T07:58:39.670254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655425+0000","last_change":"2023-11-24T07:58:37.660588+0000","last_active":"2023-11-24T07:58:39.655425+0000","last_peered":"2023-11-24T07:58:39.655425+0000","last_clean":"2023-11-24T07:58:39.655425+0000","last_became_active":"2023-11-24T07:58:37.660295+0000","last_became_peered":"2023-11-24T07:58:37.660295+0000","last_unstale":"2023-11-24T07:58:39.655425+0000","last_undegraded":"2023-11-24T07:58:39.655425+0000","last_fullsized":"2023-11-24T07:58:39.655425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:22.449742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655528+0000","last_change":"2023-11-24T07:58:38.671305+0000","last_active":"2023-11-24T07:58:39.655528+0000","last_peered":"2023-11-24T07:58:39.655528+0000","last_clean":"2023-11-24T07:58:39.655528+0000","last_became_active":"2023-11-24T07:58:38.670974+0000","last_became_peered":"2023-11-24T07:58:38.670974+0000","last_unstale":"2023-11-24T07:58:39.655528+0000","last_undegraded":"2023-11-24T07:58:39.655528+0000","last_fullsized":"2023-11-24T07:58:39.655528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661837+0000","last_change":"2023-11-24T07:58:37.658192+0000","last_active":"2023-11-24T07:58:39.661837+0000","last_peered":"2023-11-24T07:58:39.661837+0000","last_clean":"2023-11-24T07:58:39.661837+0000","last_became_active":"2023-11-24T07:58:37.657283+0000","last_became_peered":"2023-11-24T07:58:37.657283+0000","last_unstale":"2023-11-24T07:58:39.661837+0000","last_undegraded":"2023-11-24T07:58:39.661837+0000","last_fullsized":"2023-11-24T07:58:39.661837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209719+0000","last_change":"2023-11-24T07:58:38.655330+0000","last_active":"2023-11-24T07:58:40.209719+0000","last_peered":"2023-11-24T07:58:40.209719+0000","last_clean":"2023-11-24T07:58:40.209719+0000","last_became_active":"2023-11-24T07:58:38.654988+0000","last_became_peered":"2023-11-24T07:58:38.654988+0000","last_unstale":"2023-11-24T07:58:40.209719+0000","last_undegraded":"2023-11-24T07:58:40.209719+0000","last_fullsized":"2023-11-24T07:58:40.209719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661550+0000","last_change":"2023-11-24T07:58:37.663203+0000","last_active":"2023-11-24T07:58:39.661550+0000","last_peered":"2023-11-24T07:58:39.661550+0000","last_clean":"2023-11-24T07:58:39.661550+0000","last_became_active":"2023-11-24T07:58:37.662842+0000","last_became_peered":"2023-11-24T07:58:37.662842+0000","last_unstale":"2023-11-24T07:58:39.661550+0000","last_undegraded":"2023-11-24T07:58:39.661550+0000","last_fullsized":"2023-11-24T07:58:39.661550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662636+0000","last_change":"2023-11-24T07:58:38.662184+0000","last_active":"2023-11-24T07:58:39.662636+0000","last_peered":"2023-11-24T07:58:39.662636+0000","last_clean":"2023-11-24T07:58:39.662636+0000","last_became_active":"2023-11-24T07:58:38.661870+0000","last_became_peered":"2023-11-24T07:58:38.661870+0000","last_unstale":"2023-11-24T07:58:39.662636+0000","last_undegraded":"2023-11-24T07:58:39.662636+0000","last_fullsized":"2023-11-24T07:58:39.662636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662519+0000","last_change":"2023-11-24T07:58:37.663425+0000","last_active":"2023-11-24T07:58:39.662519+0000","last_peered":"2023-11-24T07:58:39.662519+0000","last_clean":"2023-11-24T07:58:39.662519+0000","last_became_active":"2023-11-24T07:58:37.663165+0000","last_became_peered":"2023-11-24T07:58:37.663165+0000","last_unstale":"2023-11-24T07:58:39.662519+0000","last_undegraded":"2023-11-24T07:58:39.662519+0000","last_fullsized":"2023-11-24T07:58:39.662519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670128+0000","last_change":"2023-11-24T07:58:38.669831+0000","last_active":"2023-11-24T07:58:39.670128+0000","last_peered":"2023-11-24T07:58:39.670128+0000","last_clean":"2023-11-24T07:58:39.670128+0000","last_became_active":"2023-11-24T07:58:38.669524+0000","last_became_peered":"2023-11-24T07:58:38.669524+0000","last_unstale":"2023-11-24T07:58:39.670128+0000","last_undegraded":"2023-11-24T07:58:39.670128+0000","last_fullsized":"2023-11-24T07:58:39.670128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:11.572131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209893+0000","last_change":"2023-11-24T07:58:37.656248+0000","last_active":"2023-11-24T07:58:40.209893+0000","last_peered":"2023-11-24T07:58:40.209893+0000","last_clean":"2023-11-24T07:58:40.209893+0000","last_became_active":"2023-11-24T07:58:37.655538+0000","last_became_peered":"2023-11-24T07:58:37.655538+0000","last_unstale":"2023-11-24T07:58:40.209893+0000","last_undegraded":"2023-11-24T07:58:40.209893+0000","last_fullsized":"2023-11-24T07:58:40.209893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:29.580446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210389+0000","last_change":"2023-11-24T07:58:38.660386+0000","last_active":"2023-11-24T07:58:40.210389+0000","last_peered":"2023-11-24T07:58:40.210389+0000","last_clean":"2023-11-24T07:58:40.210389+0000","last_became_active":"2023-11-24T07:58:38.659454+0000","last_became_peered":"2023-11-24T07:58:38.659454+0000","last_unstale":"2023-11-24T07:58:40.210389+0000","last_undegraded":"2023-11-24T07:58:40.210389+0000","last_fullsized":"2023-11-24T07:58:40.210389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661142+0000","last_change":"2023-11-24T07:58:37.663115+0000","last_active":"2023-11-24T07:58:39.661142+0000","last_peered":"2023-11-24T07:58:39.661142+0000","last_clean":"2023-11-24T07:58:39.661142+0000","last_became_active":"2023-11-24T07:58:37.662759+0000","last_became_peered":"2023-11-24T07:58:37.662759+0000","last_unstale":"2023-11-24T07:58:39.661142+0000","last_undegraded":"2023-11-24T07:58:39.661142+0000","last_fullsized":"2023-11-24T07:58:39.661142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210514+0000","last_change":"2023-11-24T07:58:38.677184+0000","last_active":"2023-11-24T07:58:40.210514+0000","last_peered":"2023-11-24T07:58:40.210514+0000","last_clean":"2023-11-24T07:58:40.210514+0000","last_became_active":"2023-11-24T07:58:38.676729+0000","last_became_peered":"2023-11-24T07:58:38.676729+0000","last_unstale":"2023-11-24T07:58:40.210514+0000","last_undegraded":"2023-11-24T07:58:40.210514+0000","last_fullsized":"2023-11-24T07:58:40.210514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670164+0000","last_change":"2023-11-24T07:58:37.659853+0000","last_active":"2023-11-24T07:58:39.670164+0000","last_peered":"2023-11-24T07:58:39.670164+0000","last_clean":"2023-11-24T07:58:39.670164+0000","last_became_active":"2023-11-24T07:58:37.659563+0000","last_became_peered":"2023-11-24T07:58:37.659563+0000","last_unstale":"2023-11-24T07:58:39.670164+0000","last_undegraded":"2023-11-24T07:58:39.670164+0000","last_fullsized":"2023-11-24T07:58:39.670164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670259+0000","last_change":"2023-11-24T07:58:38.668668+0000","last_active":"2023-11-24T07:58:39.670259+0000","last_peered":"2023-11-24T07:58:39.670259+0000","last_clean":"2023-11-24T07:58:39.670259+0000","last_became_active":"2023-11-24T07:58:38.668181+0000","last_became_peered":"2023-11-24T07:58:38.668181+0000","last_unstale":"2023-11-24T07:58:39.670259+0000","last_undegraded":"2023-11-24T07:58:39.670259+0000","last_fullsized":"2023-11-24T07:58:39.670259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662643+0000","last_change":"2023-11-24T07:58:37.656567+0000","last_active":"2023-11-24T07:58:39.662643+0000","last_peered":"2023-11-24T07:58:39.662643+0000","last_clean":"2023-11-24T07:58:39.662643+0000","last_became_active":"2023-11-24T07:58:37.656024+0000","last_became_peered":"2023-11-24T07:58:37.656024+0000","last_unstale":"2023-11-24T07:58:39.662643+0000","last_undegraded":"2023-11-24T07:58:39.662643+0000","last_fullsized":"2023-11-24T07:58:39.662643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210630+0000","last_change":"2023-11-24T07:58:38.664336+0000","last_active":"2023-11-24T07:58:40.210630+0000","last_peered":"2023-11-24T07:58:40.210630+0000","last_clean":"2023-11-24T07:58:40.210630+0000","last_became_active":"2023-11-24T07:58:38.664080+0000","last_became_peered":"2023-11-24T07:58:38.664080+0000","last_unstale":"2023-11-24T07:58:40.210630+0000","last_undegraded":"2023-11-24T07:58:40.210630+0000","last_fullsized":"2023-11-24T07:58:40.210630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.327694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660834+0000","last_change":"2023-11-24T07:58:37.660791+0000","last_active":"2023-11-24T07:58:39.660834+0000","last_peered":"2023-11-24T07:58:39.660834+0000","last_clean":"2023-11-24T07:58:39.660834+0000","last_became_active":"2023-11-24T07:58:37.660488+0000","last_became_peered":"2023-11-24T07:58:37.660488+0000","last_unstale":"2023-11-24T07:58:39.660834+0000","last_undegraded":"2023-11-24T07:58:39.660834+0000","last_fullsized":"2023-11-24T07:58:39.660834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660799+0000","last_change":"2023-11-24T07:58:38.670298+0000","last_active":"2023-11-24T07:58:39.660799+0000","last_peered":"2023-11-24T07:58:39.660799+0000","last_clean":"2023-11-24T07:58:39.660799+0000","last_became_active":"2023-11-24T07:58:38.669956+0000","last_became_peered":"2023-11-24T07:58:38.669956+0000","last_unstale":"2023-11-24T07:58:39.660799+0000","last_undegraded":"2023-11-24T07:58:39.660799+0000","last_fullsized":"2023-11-24T07:58:39.660799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209457+0000","last_change":"2023-11-24T07:58:37.659643+0000","last_active":"2023-11-24T07:58:40.209457+0000","last_peered":"2023-11-24T07:58:40.209457+0000","last_clean":"2023-11-24T07:58:40.209457+0000","last_became_active":"2023-11-24T07:58:37.659317+0000","last_became_peered":"2023-11-24T07:58:37.659317+0000","last_unstale":"2023-11-24T07:58:40.209457+0000","last_undegraded":"2023-11-24T07:58:40.209457+0000","last_fullsized":"2023-11-24T07:58:40.209457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:32.387984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209982+0000","last_change":"2023-11-24T07:58:38.667271+0000","last_active":"2023-11-24T07:58:40.209982+0000","last_peered":"2023-11-24T07:58:40.209982+0000","last_clean":"2023-11-24T07:58:40.209982+0000","last_became_active":"2023-11-24T07:58:38.666904+0000","last_became_peered":"2023-11-24T07:58:38.666904+0000","last_unstale":"2023-11-24T07:58:40.209982+0000","last_undegraded":"2023-11-24T07:58:40.209982+0000","last_fullsized":"2023-11-24T07:58:40.209982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210573+0000","last_change":"2023-11-24T07:58:37.657085+0000","last_active":"2023-11-24T07:58:40.210573+0000","last_peered":"2023-11-24T07:58:40.210573+0000","last_clean":"2023-11-24T07:58:40.210573+0000","last_became_active":"2023-11-24T07:58:37.656676+0000","last_became_peered":"2023-11-24T07:58:37.656676+0000","last_unstale":"2023-11-24T07:58:40.210573+0000","last_undegraded":"2023-11-24T07:58:40.210573+0000","last_fullsized":"2023-11-24T07:58:40.210573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:45.701317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210306+0000","last_change":"2023-11-24T07:58:38.664417+0000","last_active":"2023-11-24T07:58:40.210306+0000","last_peered":"2023-11-24T07:58:40.210306+0000","last_clean":"2023-11-24T07:58:40.210306+0000","last_became_active":"2023-11-24T07:58:38.664161+0000","last_became_peered":"2023-11-24T07:58:38.664161+0000","last_unstale":"2023-11-24T07:58:40.210306+0000","last_undegraded":"2023-11-24T07:58:40.210306+0000","last_fullsized":"2023-11-24T07:58:40.210306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:35.388773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209807+0000","last_change":"2023-11-24T07:58:37.660866+0000","last_active":"2023-11-24T07:58:40.209807+0000","last_peered":"2023-11-24T07:58:40.209807+0000","last_clean":"2023-11-24T07:58:40.209807+0000","last_became_active":"2023-11-24T07:58:37.660577+0000","last_became_peered":"2023-11-24T07:58:37.660577+0000","last_unstale":"2023-11-24T07:58:40.209807+0000","last_undegraded":"2023-11-24T07:58:40.209807+0000","last_fullsized":"2023-11-24T07:58:40.209807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662441+0000","last_change":"2023-11-24T07:58:38.674685+0000","last_active":"2023-11-24T07:58:39.662441+0000","last_peered":"2023-11-24T07:58:39.662441+0000","last_clean":"2023-11-24T07:58:39.662441+0000","last_became_active":"2023-11-24T07:58:38.674366+0000","last_became_peered":"2023-11-24T07:58:38.674366+0000","last_unstale":"2023-11-24T07:58:39.662441+0000","last_undegraded":"2023-11-24T07:58:39.662441+0000","last_fullsized":"2023-11-24T07:58:39.662441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:42.734715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662580+0000","last_change":"2023-11-24T07:58:37.655735+0000","last_active":"2023-11-24T07:58:39.662580+0000","last_peered":"2023-11-24T07:58:39.662580+0000","last_clean":"2023-11-24T07:58:39.662580+0000","last_became_active":"2023-11-24T07:58:37.655444+0000","last_became_peered":"2023-11-24T07:58:37.655444+0000","last_unstale":"2023-11-24T07:58:39.662580+0000","last_undegraded":"2023-11-24T07:58:39.662580+0000","last_fullsized":"2023-11-24T07:58:39.662580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:10.569901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210250+0000","last_change":"2023-11-24T07:58:38.667440+0000","last_active":"2023-11-24T07:58:40.210250+0000","last_peered":"2023-11-24T07:58:40.210250+0000","last_clean":"2023-11-24T07:58:40.210250+0000","last_became_active":"2023-11-24T07:58:38.666914+0000","last_became_peered":"2023-11-24T07:58:38.666914+0000","last_unstale":"2023-11-24T07:58:40.210250+0000","last_undegraded":"2023-11-24T07:58:40.210250+0000","last_fullsized":"2023-11-24T07:58:40.210250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:19.761593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209468+0000","last_change":"2023-11-24T07:58:38.655018+0000","last_active":"2023-11-24T07:58:40.209468+0000","last_peered":"2023-11-24T07:58:40.209468+0000","last_clean":"2023-11-24T07:58:40.209468+0000","last_became_active":"2023-11-24T07:58:38.654604+0000","last_became_peered":"2023-11-24T07:58:38.654604+0000","last_unstale":"2023-11-24T07:58:40.209468+0000","last_undegraded":"2023-11-24T07:58:40.209468+0000","last_fullsized":"2023-11-24T07:58:40.209468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209541+0000","last_change":"2023-11-24T07:58:37.660113+0000","last_active":"2023-11-24T07:58:40.209541+0000","last_peered":"2023-11-24T07:58:40.209541+0000","last_clean":"2023-11-24T07:58:40.209541+0000","last_became_active":"2023-11-24T07:58:37.659747+0000","last_became_peered":"2023-11-24T07:58:37.659747+0000","last_unstale":"2023-11-24T07:58:40.209541+0000","last_undegraded":"2023-11-24T07:58:40.209541+0000","last_fullsized":"2023-11-24T07:58:40.209541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:02.242362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655662+0000","last_change":"2023-11-24T07:58:38.670798+0000","last_active":"2023-11-24T07:58:39.655662+0000","last_peered":"2023-11-24T07:58:39.655662+0000","last_clean":"2023-11-24T07:58:39.655662+0000","last_became_active":"2023-11-24T07:58:38.670464+0000","last_became_peered":"2023-11-24T07:58:38.670464+0000","last_unstale":"2023-11-24T07:58:39.655662+0000","last_undegraded":"2023-11-24T07:58:39.655662+0000","last_fullsized":"2023-11-24T07:58:39.655662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662521+0000","last_change":"2023-11-24T07:58:37.661568+0000","last_active":"2023-11-24T07:58:39.662521+0000","last_peered":"2023-11-24T07:58:39.662521+0000","last_clean":"2023-11-24T07:58:39.662521+0000","last_became_active":"2023-11-24T07:58:37.661211+0000","last_became_peered":"2023-11-24T07:58:37.661211+0000","last_unstale":"2023-11-24T07:58:39.662521+0000","last_undegraded":"2023-11-24T07:58:39.662521+0000","last_fullsized":"2023-11-24T07:58:39.662521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:58:38.650810+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:58:38.650810+0000","last_peered":"2023-11-24T07:58:38.650810+0000","last_clean":"2023-11-24T07:58:38.650810+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:58:38.650810+0000","last_undegraded":"2023-11-24T07:58:38.650810+0000","last_fullsized":"2023-11-24T07:58:38.650810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.669820+0000","last_change":"2023-11-24T07:58:38.667012+0000","last_active":"2023-11-24T07:58:39.669820+0000","last_peered":"2023-11-24T07:58:39.669820+0000","last_clean":"2023-11-24T07:58:39.669820+0000","last_became_active":"2023-11-24T07:58:38.666604+0000","last_became_peered":"2023-11-24T07:58:38.666604+0000","last_unstale":"2023-11-24T07:58:39.669820+0000","last_undegraded":"2023-11-24T07:58:39.669820+0000","last_fullsized":"2023-11-24T07:58:39.669820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:11.085407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209908+0000","last_change":"2023-11-24T07:58:37.655598+0000","last_active":"2023-11-24T07:58:40.209908+0000","last_peered":"2023-11-24T07:58:40.209908+0000","last_clean":"2023-11-24T07:58:40.209908+0000","last_became_active":"2023-11-24T07:58:37.655321+0000","last_became_peered":"2023-11-24T07:58:37.655321+0000","last_unstale":"2023-11-24T07:58:40.209908+0000","last_undegraded":"2023-11-24T07:58:40.209908+0000","last_fullsized":"2023-11-24T07:58:40.209908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210609+0000","last_change":"2023-11-24T07:58:37.659615+0000","last_active":"2023-11-24T07:58:40.210609+0000","last_peered":"2023-11-24T07:58:40.210609+0000","last_clean":"2023-11-24T07:58:40.210609+0000","last_became_active":"2023-11-24T07:58:37.659362+0000","last_became_peered":"2023-11-24T07:58:37.659362+0000","last_unstale":"2023-11-24T07:58:40.210609+0000","last_undegraded":"2023-11-24T07:58:40.210609+0000","last_fullsized":"2023-11-24T07:58:40.210609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:14.569738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209722+0000","last_change":"2023-11-24T07:58:38.655615+0000","last_active":"2023-11-24T07:58:40.209722+0000","last_peered":"2023-11-24T07:58:40.209722+0000","last_clean":"2023-11-24T07:58:40.209722+0000","last_became_active":"2023-11-24T07:58:38.654516+0000","last_became_peered":"2023-11-24T07:58:38.654516+0000","last_unstale":"2023-11-24T07:58:40.209722+0000","last_undegraded":"2023-11-24T07:58:40.209722+0000","last_fullsized":"2023-11-24T07:58:40.209722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.583888+0000","last_change":"2023-11-24T07:58:37.663218+0000","last_active":"2023-11-24T07:58:41.583888+0000","last_peered":"2023-11-24T07:58:41.583888+0000","last_clean":"2023-11-24T07:58:41.583888+0000","last_became_active":"2023-11-24T07:58:37.662867+0000","last_became_peered":"2023-11-24T07:58:37.662867+0000","last_unstale":"2023-11-24T07:58:41.583888+0000","last_undegraded":"2023-11-24T07:58:41.583888+0000","last_fullsized":"2023-11-24T07:58:41.583888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656341+0000","last_change":"2023-11-24T07:58:38.670569+0000","last_active":"2023-11-24T07:58:39.656341+0000","last_peered":"2023-11-24T07:58:39.656341+0000","last_clean":"2023-11-24T07:58:39.656341+0000","last_became_active":"2023-11-24T07:58:38.670239+0000","last_became_peered":"2023-11-24T07:58:38.670239+0000","last_unstale":"2023-11-24T07:58:39.656341+0000","last_undegraded":"2023-11-24T07:58:39.656341+0000","last_fullsized":"2023-11-24T07:58:39.656341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661516+0000","last_change":"2023-11-24T07:58:37.667212+0000","last_active":"2023-11-24T07:58:39.661516+0000","last_peered":"2023-11-24T07:58:39.661516+0000","last_clean":"2023-11-24T07:58:39.661516+0000","last_became_active":"2023-11-24T07:58:37.666944+0000","last_became_peered":"2023-11-24T07:58:37.666944+0000","last_unstale":"2023-11-24T07:58:39.661516+0000","last_undegraded":"2023-11-24T07:58:39.661516+0000","last_fullsized":"2023-11-24T07:58:39.661516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:50.810778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670071+0000","last_change":"2023-11-24T07:58:38.660314+0000","last_active":"2023-11-24T07:58:39.670071+0000","last_peered":"2023-11-24T07:58:39.670071+0000","last_clean":"2023-11-24T07:58:39.670071+0000","last_became_active":"2023-11-24T07:58:38.660005+0000","last_became_peered":"2023-11-24T07:58:38.660005+0000","last_unstale":"2023-11-24T07:58:39.670071+0000","last_undegraded":"2023-11-24T07:58:39.670071+0000","last_fullsized":"2023-11-24T07:58:39.670071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209750+0000","last_change":"2023-11-24T07:58:37.667045+0000","last_active":"2023-11-24T07:58:40.209750+0000","last_peered":"2023-11-24T07:58:40.209750+0000","last_clean":"2023-11-24T07:58:40.209750+0000","last_became_active":"2023-11-24T07:58:37.666766+0000","last_became_peered":"2023-11-24T07:58:37.666766+0000","last_unstale":"2023-11-24T07:58:40.209750+0000","last_undegraded":"2023-11-24T07:58:40.209750+0000","last_fullsized":"2023-11-24T07:58:40.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209814+0000","last_change":"2023-11-24T07:58:38.652046+0000","last_active":"2023-11-24T07:58:40.209814+0000","last_peered":"2023-11-24T07:58:40.209814+0000","last_clean":"2023-11-24T07:58:40.209814+0000","last_became_active":"2023-11-24T07:58:38.651762+0000","last_became_peered":"2023-11-24T07:58:38.651762+0000","last_unstale":"2023-11-24T07:58:40.209814+0000","last_undegraded":"2023-11-24T07:58:40.209814+0000","last_fullsized":"2023-11-24T07:58:40.209814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:28.237034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209556+0000","last_change":"2023-11-24T07:58:37.655838+0000","last_active":"2023-11-24T07:58:40.209556+0000","last_peered":"2023-11-24T07:58:40.209556+0000","last_clean":"2023-11-24T07:58:40.209556+0000","last_became_active":"2023-11-24T07:58:37.655553+0000","last_became_peered":"2023-11-24T07:58:37.655553+0000","last_unstale":"2023-11-24T07:58:40.209556+0000","last_undegraded":"2023-11-24T07:58:40.209556+0000","last_fullsized":"2023-11-24T07:58:40.209556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:24.121557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655413+0000","last_change":"2023-11-24T07:58:38.655539+0000","last_active":"2023-11-24T07:58:39.655413+0000","last_peered":"2023-11-24T07:58:39.655413+0000","last_clean":"2023-11-24T07:58:39.655413+0000","last_became_active":"2023-11-24T07:58:38.655169+0000","last_became_peered":"2023-11-24T07:58:38.655169+0000","last_unstale":"2023-11-24T07:58:39.655413+0000","last_undegraded":"2023-11-24T07:58:39.655413+0000","last_fullsized":"2023-11-24T07:58:39.655413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:30.582676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209712+0000","last_change":"2023-11-24T07:58:37.655841+0000","last_active":"2023-11-24T07:58:40.209712+0000","last_peered":"2023-11-24T07:58:40.209712+0000","last_clean":"2023-11-24T07:58:40.209712+0000","last_became_active":"2023-11-24T07:58:37.655487+0000","last_became_peered":"2023-11-24T07:58:37.655487+0000","last_unstale":"2023-11-24T07:58:40.209712+0000","last_undegraded":"2023-11-24T07:58:40.209712+0000","last_fullsized":"2023-11-24T07:58:40.209712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.102055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655699+0000","last_change":"2023-11-24T07:58:38.671051+0000","last_active":"2023-11-24T07:58:39.655699+0000","last_peered":"2023-11-24T07:58:39.655699+0000","last_clean":"2023-11-24T07:58:39.655699+0000","last_became_active":"2023-11-24T07:58:38.670571+0000","last_became_peered":"2023-11-24T07:58:38.670571+0000","last_unstale":"2023-11-24T07:58:39.655699+0000","last_undegraded":"2023-11-24T07:58:39.655699+0000","last_fullsized":"2023-11-24T07:58:39.655699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210427+0000","last_change":"2023-11-24T07:58:37.655295+0000","last_active":"2023-11-24T07:58:40.210427+0000","last_peered":"2023-11-24T07:58:40.210427+0000","last_clean":"2023-11-24T07:58:40.210427+0000","last_became_active":"2023-11-24T07:58:37.654767+0000","last_became_peered":"2023-11-24T07:58:37.654767+0000","last_unstale":"2023-11-24T07:58:40.210427+0000","last_undegraded":"2023-11-24T07:58:40.210427+0000","last_fullsized":"2023-11-24T07:58:40.210427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:03.074227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.660877+0000","last_change":"2023-11-24T07:58:38.662792+0000","last_active":"2023-11-24T07:58:39.660877+0000","last_peered":"2023-11-24T07:58:39.660877+0000","last_clean":"2023-11-24T07:58:39.660877+0000","last_became_active":"2023-11-24T07:58:38.662492+0000","last_became_peered":"2023-11-24T07:58:38.662492+0000","last_unstale":"2023-11-24T07:58:39.660877+0000","last_undegraded":"2023-11-24T07:58:39.660877+0000","last_fullsized":"2023-11-24T07:58:39.660877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:43.148975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210083+0000","last_change":"2023-11-24T07:58:37.660634+0000","last_active":"2023-11-24T07:58:40.210083+0000","last_peered":"2023-11-24T07:58:40.210083+0000","last_clean":"2023-11-24T07:58:40.210083+0000","last_became_active":"2023-11-24T07:58:37.660340+0000","last_became_peered":"2023-11-24T07:58:37.660340+0000","last_unstale":"2023-11-24T07:58:40.210083+0000","last_undegraded":"2023-11-24T07:58:40.210083+0000","last_fullsized":"2023-11-24T07:58:40.210083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:04.555457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210612+0000","last_change":"2023-11-24T07:58:38.672774+0000","last_active":"2023-11-24T07:58:40.210612+0000","last_peered":"2023-11-24T07:58:40.210612+0000","last_clean":"2023-11-24T07:58:40.210612+0000","last_became_active":"2023-11-24T07:58:38.672498+0000","last_became_peered":"2023-11-24T07:58:38.672498+0000","last_unstale":"2023-11-24T07:58:40.210612+0000","last_undegraded":"2023-11-24T07:58:40.210612+0000","last_fullsized":"2023-11-24T07:58:40.210612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:49.619201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209839+0000","last_change":"2023-11-24T07:58:37.653547+0000","last_active":"2023-11-24T07:58:40.209839+0000","last_peered":"2023-11-24T07:58:40.209839+0000","last_clean":"2023-11-24T07:58:40.209839+0000","last_became_active":"2023-11-24T07:58:37.653148+0000","last_became_peered":"2023-11-24T07:58:37.653148+0000","last_unstale":"2023-11-24T07:58:40.209839+0000","last_undegraded":"2023-11-24T07:58:40.209839+0000","last_fullsized":"2023-11-24T07:58:40.209839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:10.083177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662278+0000","last_change":"2023-11-24T07:58:38.676919+0000","last_active":"2023-11-24T07:58:39.662278+0000","last_peered":"2023-11-24T07:58:39.662278+0000","last_clean":"2023-11-24T07:58:39.662278+0000","last_became_active":"2023-11-24T07:58:38.676622+0000","last_became_peered":"2023-11-24T07:58:38.676622+0000","last_unstale":"2023-11-24T07:58:39.662278+0000","last_undegraded":"2023-11-24T07:58:39.662278+0000","last_fullsized":"2023-11-24T07:58:39.662278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.425444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210143+0000","last_change":"2023-11-24T07:58:37.659572+0000","last_active":"2023-11-24T07:58:40.210143+0000","last_peered":"2023-11-24T07:58:40.210143+0000","last_clean":"2023-11-24T07:58:40.210143+0000","last_became_active":"2023-11-24T07:58:37.659273+0000","last_became_peered":"2023-11-24T07:58:37.659273+0000","last_unstale":"2023-11-24T07:58:40.210143+0000","last_undegraded":"2023-11-24T07:58:40.210143+0000","last_fullsized":"2023-11-24T07:58:40.210143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:36.630556+0000","last_clean_scrub_stamp":"2023-11-24T07:58:36.630556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:18.759363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.210569+0000","last_change":"2023-11-24T07:58:38.673691+0000","last_active":"2023-11-24T07:58:40.210569+0000","last_peered":"2023-11-24T07:58:40.210569+0000","last_clean":"2023-11-24T07:58:40.210569+0000","last_became_active":"2023-11-24T07:58:38.673092+0000","last_became_peered":"2023-11-24T07:58:38.673092+0000","last_unstale":"2023-11-24T07:58:40.210569+0000","last_undegraded":"2023-11-24T07:58:40.210569+0000","last_fullsized":"2023-11-24T07:58:40.210569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:11.089038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661147+0000","last_change":"2023-11-24T07:58:38.670271+0000","last_active":"2023-11-24T07:58:39.661147+0000","last_peered":"2023-11-24T07:58:39.661147+0000","last_clean":"2023-11-24T07:58:39.661147+0000","last_became_active":"2023-11-24T07:58:38.669965+0000","last_became_peered":"2023-11-24T07:58:38.669965+0000","last_unstale":"2023-11-24T07:58:39.661147+0000","last_undegraded":"2023-11-24T07:58:39.661147+0000","last_fullsized":"2023-11-24T07:58:39.661147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209683+0000","last_change":"2023-11-24T07:58:38.663275+0000","last_active":"2023-11-24T07:58:40.209683+0000","last_peered":"2023-11-24T07:58:40.209683+0000","last_clean":"2023-11-24T07:58:40.209683+0000","last_became_active":"2023-11-24T07:58:38.662928+0000","last_became_peered":"2023-11-24T07:58:38.662928+0000","last_unstale":"2023-11-24T07:58:40.209683+0000","last_undegraded":"2023-11-24T07:58:40.209683+0000","last_fullsized":"2023-11-24T07:58:40.209683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:23.451972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662492+0000","last_change":"2023-11-24T07:58:38.674085+0000","last_active":"2023-11-24T07:58:39.662492+0000","last_peered":"2023-11-24T07:58:39.662492+0000","last_clean":"2023-11-24T07:58:39.662492+0000","last_became_active":"2023-11-24T07:58:38.673780+0000","last_became_peered":"2023-11-24T07:58:38.673780+0000","last_unstale":"2023-11-24T07:58:39.662492+0000","last_undegraded":"2023-11-24T07:58:39.662492+0000","last_fullsized":"2023-11-24T07:58:39.662492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:36.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670095+0000","last_change":"2023-11-24T07:58:38.676742+0000","last_active":"2023-11-24T07:58:39.670095+0000","last_peered":"2023-11-24T07:58:39.670095+0000","last_clean":"2023-11-24T07:58:39.670095+0000","last_became_active":"2023-11-24T07:58:38.676422+0000","last_became_peered":"2023-11-24T07:58:38.676422+0000","last_unstale":"2023-11-24T07:58:39.670095+0000","last_undegraded":"2023-11-24T07:58:39.670095+0000","last_fullsized":"2023-11-24T07:58:39.670095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:26.034740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655920+0000","last_change":"2023-11-24T07:58:38.674946+0000","last_active":"2023-11-24T07:58:39.655920+0000","last_peered":"2023-11-24T07:58:39.655920+0000","last_clean":"2023-11-24T07:58:39.655920+0000","last_became_active":"2023-11-24T07:58:38.674458+0000","last_became_peered":"2023-11-24T07:58:38.674458+0000","last_unstale":"2023-11-24T07:58:39.655920+0000","last_undegraded":"2023-11-24T07:58:39.655920+0000","last_fullsized":"2023-11-24T07:58:39.655920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:40.209765+0000","last_change":"2023-11-24T07:58:38.656486+0000","last_active":"2023-11-24T07:58:40.209765+0000","last_peered":"2023-11-24T07:58:40.209765+0000","last_clean":"2023-11-24T07:58:40.209765+0000","last_became_active":"2023-11-24T07:58:38.654844+0000","last_became_peered":"2023-11-24T07:58:38.654844+0000","last_unstale":"2023-11-24T07:58:40.209765+0000","last_undegraded":"2023-11-24T07:58:40.209765+0000","last_fullsized":"2023-11-24T07:58:40.209765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661231+0000","last_change":"2023-11-24T07:58:38.666517+0000","last_active":"2023-11-24T07:58:39.661231+0000","last_peered":"2023-11-24T07:58:39.661231+0000","last_clean":"2023-11-24T07:58:39.661231+0000","last_became_active":"2023-11-24T07:58:38.665821+0000","last_became_peered":"2023-11-24T07:58:38.665821+0000","last_unstale":"2023-11-24T07:58:39.661231+0000","last_undegraded":"2023-11-24T07:58:39.661231+0000","last_fullsized":"2023-11-24T07:58:39.661231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.656201+0000","last_change":"2023-11-24T07:58:38.672413+0000","last_active":"2023-11-24T07:58:39.656201+0000","last_peered":"2023-11-24T07:58:39.656201+0000","last_clean":"2023-11-24T07:58:39.656201+0000","last_became_active":"2023-11-24T07:58:38.672131+0000","last_became_peered":"2023-11-24T07:58:38.672131+0000","last_unstale":"2023-11-24T07:58:39.656201+0000","last_undegraded":"2023-11-24T07:58:39.656201+0000","last_fullsized":"2023-11-24T07:58:39.656201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662001+0000","last_change":"2023-11-24T07:58:38.672368+0000","last_active":"2023-11-24T07:58:39.662001+0000","last_peered":"2023-11-24T07:58:39.662001+0000","last_clean":"2023-11-24T07:58:39.662001+0000","last_became_active":"2023-11-24T07:58:38.672038+0000","last_became_peered":"2023-11-24T07:58:38.672038+0000","last_unstale":"2023-11-24T07:58:39.662001+0000","last_undegraded":"2023-11-24T07:58:39.662001+0000","last_fullsized":"2023-11-24T07:58:39.662001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:38.707028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.655743+0000","last_change":"2023-11-24T07:58:38.671825+0000","last_active":"2023-11-24T07:58:39.655743+0000","last_peered":"2023-11-24T07:58:39.655743+0000","last_clean":"2023-11-24T07:58:39.655743+0000","last_became_active":"2023-11-24T07:58:38.671530+0000","last_became_peered":"2023-11-24T07:58:38.671530+0000","last_unstale":"2023-11-24T07:58:39.655743+0000","last_undegraded":"2023-11-24T07:58:39.655743+0000","last_fullsized":"2023-11-24T07:58:39.655743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:57.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.662135+0000","last_change":"2023-11-24T07:58:38.663149+0000","last_active":"2023-11-24T07:58:39.662135+0000","last_peered":"2023-11-24T07:58:39.662135+0000","last_clean":"2023-11-24T07:58:39.662135+0000","last_became_active":"2023-11-24T07:58:38.662826+0000","last_became_peered":"2023-11-24T07:58:38.662826+0000","last_unstale":"2023-11-24T07:58:39.662135+0000","last_undegraded":"2023-11-24T07:58:39.662135+0000","last_fullsized":"2023-11-24T07:58:39.662135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:44.287001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.670177+0000","last_change":"2023-11-24T07:58:38.673409+0000","last_active":"2023-11-24T07:58:39.670177+0000","last_peered":"2023-11-24T07:58:39.670177+0000","last_clean":"2023-11-24T07:58:39.670177+0000","last_became_active":"2023-11-24T07:58:38.673109+0000","last_became_peered":"2023-11-24T07:58:38.673109+0000","last_unstale":"2023-11-24T07:58:39.670177+0000","last_undegraded":"2023-11-24T07:58:39.670177+0000","last_fullsized":"2023-11-24T07:58:39.670177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:15.571968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T07:58:39.661303+0000","last_change":"2023-11-24T07:58:38.666773+0000","last_active":"2023-11-24T07:58:39.661303+0000","last_peered":"2023-11-24T07:58:39.661303+0000","last_clean":"2023-11-24T07:58:39.661303+0000","last_became_active":"2023-11-24T07:58:38.666430+0000","last_became_peered":"2023-11-24T07:58:38.666430+0000","last_unstale":"2023-11-24T07:58:39.661303+0000","last_undegraded":"2023-11-24T07:58:39.661303+0000","last_fullsized":"2023-11-24T07:58:39.661303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:37.632786+0000","last_clean_scrub_stamp":"2023-11-24T07:58:37.632786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:45.568105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:48.013 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T07:58:48.013 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T07:58:48.014 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T07:58:48.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T07:58:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.171+0000 7f728775b700 1 Processor -- start 2023-11-24T07:58:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.172+0000 7f728775b700 1 -- start start 2023-11-24T07:58:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.172+0000 7f728775b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f72800b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.172+0000 7f728775b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72800af4d0 0x7f72800afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.172+0000 7f728775b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800ae430 con 0x7f728015e080 2023-11-24T07:58:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.172+0000 7f728775b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72800af0b0 con 0x7f72800af4d0 2023-11-24T07:58:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.172+0000 7f728775b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800b05a0 con 0x7f728006c710 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7284cf6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f72800b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7285cf8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72800af4d0 0x7f72800afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7284cf6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f72800b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:52154/0 (socket says 172.21.15.32:52154) 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7284cf6700 1 -- 172.21.15.32:0/2970985612 learned_addr learned my addr 172.21.15.32:0/2970985612 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7285cf8700 1 -- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 msgr2=0x7f72800b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7285cf8700 1 --2- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f72800b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7285cf8700 1 -- 172.21.15.32:0/2970985612 >> v1:172.21.15.123:6789/0 conn(0x7f728015e080 legacy=0x7f728006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7285cf8700 1 -- 172.21.15.32:0/2970985612 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72800ae710 con 0x7f72800af4d0 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7284cf6700 1 --2- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f72800b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7285cf8700 1 --2- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f72800afb90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7278009dc0 tx=0x7f727800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87cf1a243c7d17f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.173+0000 7f7273fff700 1 -- 172.21.15.32:0/2970985612 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800e070 con 0x7f72800af4d0 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f7273fff700 1 -- 172.21.15.32:0/2970985612 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800bae0 con 0x7f72800af4d0 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f7273fff700 1 -- 172.21.15.32:0/2970985612 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800be90 con 0x7f72800af4d0 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 -- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 msgr2=0x7f72800afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 --2- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f72800afb90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7278009dc0 tx=0x7f727800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 -- 172.21.15.32:0/2970985612 shutdown_connections 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 --2- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f72800afb90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 --2- 172.21.15.32:0/2970985612 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f72800b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 -- 172.21.15.32:0/2970985612 >> 172.21.15.32:0/2970985612 conn(0x7f7280071b90 msgr2=0x7f7280071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 -- 172.21.15.32:0/2970985612 shutdown_connections 2023-11-24T07:58:48.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.174+0000 7f728775b700 1 -- 172.21.15.32:0/2970985612 wait complete. 2023-11-24T07:58:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.175+0000 7f728775b700 1 Processor -- start 2023-11-24T07:58:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.175+0000 7f728775b700 1 -- start start 2023-11-24T07:58:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f728775b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f7280141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f728775b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f7280144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f728775b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 0x7f72800b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f728775b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72801452e0 con 0x7f728015e080 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f728775b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72801598d0 con 0x7f72800af4d0 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f728775b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280159bb0 con 0x7f728006c710 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f7285cf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 0x7f72800b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f7285cf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 0x7f72800b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56910/0 (socket says 172.21.15.32:56910) 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f7285cf8700 1 -- 172.21.15.32:0/1175157595 learned_addr learned my addr 172.21.15.32:0/1175157595 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f7284cf6700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f7280144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.176+0000 7f72854f7700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f7280141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7285cf8700 1 -- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 msgr2=0x7f7280141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7285cf8700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f7280141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7285cf8700 1 -- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 msgr2=0x7f7280144da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7285cf8700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f7280144da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7285cf8700 1 -- 172.21.15.32:0/1175157595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274009550 con 0x7f728015e080 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7285cf8700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 0x7f72800b0c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f727800b370 tx=0x7f7278004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278004550 con 0x7f728015e080 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278020070 con 0x7f728015e080 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7278009a70 con 0x7f728015e080 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278004910 con 0x7f728015e080 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.177+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728015fdc0 con 0x7f728015e080 2023-11-24T07:58:48.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.178+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72800753f0 con 0x7f728015e080 2023-11-24T07:58:48.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.179+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7278012430 con 0x7f728015e080 2023-11-24T07:58:48.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.179+0000 7f7271ffb700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f726406aed0 0x7f726406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.179+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7278022070 con 0x7f728015e080 2023-11-24T07:58:48.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.179+0000 7f72854f7700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f726406aed0 0x7f726406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.180+0000 7f72854f7700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f726406aed0 0x7f726406d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f727c007fd0 tx=0x7f727c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.183+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f727805a230 con 0x7f728015e080 2023-11-24T07:58:48.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.432+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f728015ffb0 con 0x7f728015e080 2023-11-24T07:58:48.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.432+0000 7f7271ffb700 1 -- 172.21.15.32:0/1175157595 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7278059dc0 con 0x7f728015e080 2023-11-24T07:58:48.434 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:48.434 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T07:58:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f726406aed0 msgr2=0x7f726406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f726406aed0 0x7f726406d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f727c007fd0 tx=0x7f727c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 msgr2=0x7f72800b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 0x7f72800b0c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f727800b370 tx=0x7f7278004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 shutdown_connections 2023-11-24T07:58:48.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f726406aed0 0x7f726406d390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728015e080 0x7f72800b0c50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f728006c710 0x7f7280141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 --2- 172.21.15.32:0/1175157595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800af4d0 0x7f7280144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 >> 172.21.15.32:0/1175157595 conn(0x7f7280071b90 msgr2=0x7f728006b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:48.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 shutdown_connections 2023-11-24T07:58:48.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.434+0000 7f728775b700 1 -- 172.21.15.32:0/1175157595 wait complete. 2023-11-24T07:58:48.448 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T07:58:48.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:48.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.571+0000 7f1397271700 1 Processor -- start 2023-11-24T07:58:48.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.571+0000 7f1397271700 1 -- start start 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f1397271700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f139015e1e0 0x7f13900d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f1397271700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13900d3a00 0x7f13900d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f1397271700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13900d20b0 con 0x7f13900d0db0 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f1397271700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13900d3150 con 0x7f13900d3a00 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f1397271700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900d1240 con 0x7f139015e1e0 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13900d3a00 0x7f13900d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13900d3a00 0x7f13900d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56916/0 (socket says 172.21.15.32:56916) 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 -- 172.21.15.32:0/1857963213 learned_addr learned my addr 172.21.15.32:0/1857963213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:48.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139500d700 1 --2- 172.21.15.32:0/1857963213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f139015e1e0 0x7f13900d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 -- 172.21.15.32:0/1857963213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f139015e1e0 msgr2=0x7f13900d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 --2- 172.21.15.32:0/1857963213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f139015e1e0 0x7f13900d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 -- 172.21.15.32:0/1857963213 >> v1:172.21.15.123:6789/0 conn(0x7f13900d0db0 legacy=0x7f139006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 -- 172.21.15.32:0/1857963213 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900d2390 con 0x7f13900d3a00 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.572+0000 7f139480c700 1 --2- 172.21.15.32:0/1857963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13900d3a00 0x7f13900d0870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f13800098f0 tx=0x7f1380009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ec010d30c448a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1387fff700 1 -- 172.21.15.32:0/1857963213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000b8a0 con 0x7f13900d3a00 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1387fff700 1 -- 172.21.15.32:0/1857963213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000ba00 con 0x7f13900d3a00 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1387fff700 1 -- 172.21.15.32:0/1857963213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000bd90 con 0x7f13900d3a00 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 -- 172.21.15.32:0/1857963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13900d3a00 msgr2=0x7f13900d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 --2- 172.21.15.32:0/1857963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13900d3a00 0x7f13900d0870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f13800098f0 tx=0x7f1380009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:48.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 -- 172.21.15.32:0/1857963213 shutdown_connections 2023-11-24T07:58:48.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 --2- 172.21.15.32:0/1857963213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f139015e1e0 0x7f13900d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 --2- 172.21.15.32:0/1857963213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13900d3a00 0x7f13900d0870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 -- 172.21.15.32:0/1857963213 >> 172.21.15.32:0/1857963213 conn(0x7f13900b5b50 msgr2=0x7f13900b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:48.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 -- 172.21.15.32:0/1857963213 shutdown_connections 2023-11-24T07:58:48.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.573+0000 7f1397271700 1 -- 172.21.15.32:0/1857963213 wait complete. 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.574+0000 7f1397271700 1 Processor -- start 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 -- start start 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13900d0db0 0x7f13900caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13900d3a00 0x7f13900c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 0x7f13901f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900c9490 con 0x7f139015e1e0 2023-11-24T07:58:48.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390158a00 con 0x7f13900d0db0 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f1397271700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1390158ca0 con 0x7f13900d3a00 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f139580e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 0x7f13901f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f139580e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 0x7f13901f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56924/0 (socket says 172.21.15.32:56924) 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f139580e700 1 -- 172.21.15.32:0/3070426649 learned_addr learned my addr 172.21.15.32:0/3070426649 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f139480c700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13900d3a00 0x7f13900c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.575+0000 7f139500d700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13900d0db0 0x7f13900caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f139580e700 1 -- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13900d3a00 msgr2=0x7f13900c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f139580e700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13900d3a00 0x7f13900c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f139580e700 1 -- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13900d0db0 msgr2=0x7f13900caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f139580e700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13900d0db0 0x7f13900caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f139580e700 1 -- 172.21.15.32:0/3070426649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380009550 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f139580e700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 0x7f13901f6b90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f138800cc50 tx=0x7f138800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388003c00 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1388003d60 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13900c8920 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138800aab0 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.576+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13900c91b0 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.577+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13900751e0 con 0x7f139015e1e0 2023-11-24T07:58:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.578+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f138800ac10 con 0x7f139015e1e0 2023-11-24T07:58:48.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.578+0000 7f1385ffb700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f137406aed0 0x7f137406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.578+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1388089510 con 0x7f139015e1e0 2023-11-24T07:58:48.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.578+0000 7f139500d700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f137406aed0 0x7f137406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.579+0000 7f139500d700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f137406aed0 0x7f137406d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13900c96a0 tx=0x7f138c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.582+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1388054650 con 0x7f139015e1e0 2023-11-24T07:58:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.784+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13900b6a00 con 0x7f139015e1e0 2023-11-24T07:58:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.785+0000 7f1385ffb700 1 -- 172.21.15.32:0/3070426649 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f13900b6a00 con 0x7f139015e1e0 2023-11-24T07:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f137406aed0 msgr2=0x7f137406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f137406aed0 0x7f137406d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13900c96a0 tx=0x7f138c008040 comp rx=0 tx=0).stop 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 msgr2=0x7f13901f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 0x7f13901f6b90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f138800cc50 tx=0x7f138800cf60 comp rx=0 tx=0).stop 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 shutdown_connections 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f137406aed0 0x7f137406d390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f139015e1e0 0x7f13901f6b90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13900d0db0 0x7f13900caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 --2- 172.21.15.32:0/3070426649 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13900d3a00 0x7f13900c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 >> 172.21.15.32:0/3070426649 conn(0x7f13900b5b50 msgr2=0x7f13900ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 shutdown_connections 2023-11-24T07:58:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.787+0000 7f1397271700 1 -- 172.21.15.32:0/3070426649 wait complete. 2023-11-24T07:58:48.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:48.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.927+0000 7fe9adb73700 1 Processor -- start 2023-11-24T07:58:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9adb73700 1 -- start start 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9adb73700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80746a0 0x7fe9a80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9adb73700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9a80afc90 0x7fe9a806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9adb73700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a80b3c60 con 0x7fe9a815e160 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9adb73700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a80b4500 con 0x7fe9a80afc90 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9adb73700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80b58c0 con 0x7fe9a80746a0 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.928+0000 7fe9a7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9a80afc90 0x7fe9a806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe9a815e160 0x7fe9a8073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58640/0 (socket says 172.21.15.32:58640) 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9a80afc90 0x7fe9a806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56932/0 (socket says 172.21.15.32:56932) 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 -- 172.21.15.32:0/2077188755 learned_addr learned my addr 172.21.15.32:0/2077188755 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a6ffd700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80746a0 0x7fe9a80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 -- 172.21.15.32:0/2077188755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80746a0 msgr2=0x7fe9a80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80746a0 0x7fe9a80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 -- 172.21.15.32:0/2077188755 >> v1:172.21.15.32:6789/0 conn(0x7fe9a815e160 legacy=0x7fe9a8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 -- 172.21.15.32:0/2077188755 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80b0140 con 0x7fe9a80afc90 2023-11-24T07:58:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a67fc700 1 -- 172.21.15.32:0/2077188755 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884708904 0 0) 0x7fe9a80b3c60 con 0x7fe9a815e160 2023-11-24T07:58:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a6ffd700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80746a0 0x7fe9a80b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a7fff700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a80afc90 0x7fe9a806d930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009e50 tx=0x7fe9a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d6a15f469fdfad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a67fc700 1 -- 172.21.15.32:0/2077188755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000e070 con 0x7fe9a80afc90 2023-11-24T07:58:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a67fc700 1 -- 172.21.15.32:0/2077188755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000b9c0 con 0x7fe9a80afc90 2023-11-24T07:58:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.929+0000 7fe9a67fc700 1 -- 172.21.15.32:0/2077188755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000bd70 con 0x7fe9a80afc90 2023-11-24T07:58:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 -- 172.21.15.32:0/2077188755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a80afc90 msgr2=0x7fe9a806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a80afc90 0x7fe9a806d930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009e50 tx=0x7fe9a000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 -- 172.21.15.32:0/2077188755 shutdown_connections 2023-11-24T07:58:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a80afc90 0x7fe9a806d930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 --2- 172.21.15.32:0/2077188755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80746a0 0x7fe9a80b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 -- 172.21.15.32:0/2077188755 >> 172.21.15.32:0/2077188755 conn(0x7fe9a80d0bf0 msgr2=0x7fe9a80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 -- 172.21.15.32:0/2077188755 shutdown_connections 2023-11-24T07:58:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.930+0000 7fe9adb73700 1 -- 172.21.15.32:0/2077188755 wait complete. 2023-11-24T07:58:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.931+0000 7fe9adb73700 1 Processor -- start 2023-11-24T07:58:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.931+0000 7fe9adb73700 1 -- start start 2023-11-24T07:58:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9adb73700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a80746a0 0x7fe9a814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9adb73700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80afc90 0x7fe9a8154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9adb73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 0x7fe9a8152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9adb73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a81542a0 con 0x7fe9a815e160 2023-11-24T07:58:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9adb73700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a81534a0 con 0x7fe9a80746a0 2023-11-24T07:58:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9adb73700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a815ebd0 con 0x7fe9a80afc90 2023-11-24T07:58:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 0x7fe9a8152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 0x7fe9a8152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56938/0 (socket says 172.21.15.32:56938) 2023-11-24T07:58:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 -- 172.21.15.32:0/4087091803 learned_addr learned my addr 172.21.15.32:0/4087091803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a77fe700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a80746a0 0x7fe9a814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a6ffd700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80afc90 0x7fe9a8154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 -- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80afc90 msgr2=0x7fe9a8154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80afc90 0x7fe9a8154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 -- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a80746a0 msgr2=0x7fe9a814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a80746a0 0x7fe9a814faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 -- 172.21.15.32:0/4087091803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0009b30 con 0x7fe9a815e160 2023-11-24T07:58:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.932+0000 7fe9a7fff700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 0x7fe9a8152f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000b730 tx=0x7fe9a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.933+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00092e0 con 0x7fe9a815e160 2023-11-24T07:58:48.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.933+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a001c440 con 0x7fe9a815e160 2023-11-24T07:58:48.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.933+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a8153720 con 0x7fe9a815e160 2023-11-24T07:58:48.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.933+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a001c810 con 0x7fe9a815e160 2023-11-24T07:58:48.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.933+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a8153ce0 con 0x7fe9a815e160 2023-11-24T07:58:48.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.934+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a80b4c60 con 0x7fe9a815e160 2023-11-24T07:58:48.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.934+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9a001c970 con 0x7fe9a815e160 2023-11-24T07:58:48.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.934+0000 7fe99ffff700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe98806aed0 0x7fe98806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.934+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9a008b070 con 0x7fe9a815e160 2023-11-24T07:58:48.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.935+0000 7fe9a77fe700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe98806aed0 0x7fe98806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.935+0000 7fe9a77fe700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe98806aed0 0x7fe98806d390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe998000f60 tx=0x7fe998007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:48.939+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9a00561b0 con 0x7fe9a815e160 2023-11-24T07:58:49.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.139+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9a815ed90 con 0x7fe9a815e160 2023-11-24T07:58:49.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.140+0000 7fe99ffff700 1 -- 172.21.15.32:0/4087091803 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9a00597d0 con 0x7fe9a815e160 2023-11-24T07:58:49.142 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:58:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe98806aed0 msgr2=0x7fe98806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe98806aed0 0x7fe98806d390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe998000f60 tx=0x7fe998007c90 comp rx=0 tx=0).stop 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 msgr2=0x7fe9a8152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 0x7fe9a8152f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000b730 tx=0x7fe9a0004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 shutdown_connections 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe98806aed0 0x7fe98806d390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9a815e160 0x7fe9a8152f60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a80746a0 0x7fe9a814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 --2- 172.21.15.32:0/4087091803 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9a80afc90 0x7fe9a8154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 >> 172.21.15.32:0/4087091803 conn(0x7fe9a80d0bf0 msgr2=0x7fe9a81623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 shutdown_connections 2023-11-24T07:58:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.142+0000 7fe9adb73700 1 -- 172.21.15.32:0/4087091803 wait complete. 2023-11-24T07:58:49.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.281+0000 7fc2d41a2700 1 Processor -- start 2023-11-24T07:58:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.281+0000 7fc2d41a2700 1 -- start start 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d41a2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc2cc15e290 0x7fc2cc0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d41a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc0ad870 0x7fc2cc06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d41a2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2cc06e400 con 0x7fc2cc0aedc0 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d41a2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2cc06e630 con 0x7fc2cc15e290 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d41a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc06e910 con 0x7fc2cc0ad870 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d1f3e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2cc0aedc0 0x7fc2cc0af1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58654/0 (socket says 172.21.15.32:58654) 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.282+0000 7fc2d1f3e700 1 -- 172.21.15.32:0/3399552975 learned_addr learned my addr 172.21.15.32:0/3399552975 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d273f700 1 --2- 172.21.15.32:0/3399552975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc0ad870 0x7fc2cc06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d173d700 1 --2- 172.21.15.32:0/3399552975 >> v2:172.21.15.123:3300/0 conn(0x7fc2cc15e290 0x7fc2cc0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d0f3c700 1 -- 172.21.15.32:0/3399552975 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410286848 0 0) 0x7fc2cc06e400 con 0x7fc2cc0aedc0 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d0f3c700 1 -- 172.21.15.32:0/3399552975 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b80035b0 con 0x7fc2cc0aedc0 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d273f700 1 -- 172.21.15.32:0/3399552975 >> v2:172.21.15.123:3300/0 conn(0x7fc2cc15e290 msgr2=0x7fc2cc0ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d273f700 1 --2- 172.21.15.32:0/3399552975 >> v2:172.21.15.123:3300/0 conn(0x7fc2cc15e290 0x7fc2cc0ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d273f700 1 -- 172.21.15.32:0/3399552975 >> v1:172.21.15.32:6789/0 conn(0x7fc2cc0aedc0 legacy=0x7fc2cc0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d273f700 1 -- 172.21.15.32:0/3399552975 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc06ebf0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d173d700 1 --2- 172.21.15.32:0/3399552975 >> v2:172.21.15.123:3300/0 conn(0x7fc2cc15e290 0x7fc2cc0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d273f700 1 --2- 172.21.15.32:0/3399552975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc0ad870 0x7fc2cc06f6d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009d90 tx=0x7fc2c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46efa0f720cfd40 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d0f3c700 1 -- 172.21.15.32:0/3399552975 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400e070 con 0x7fc2cc0ad870 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d0f3c700 1 -- 172.21.15.32:0/3399552975 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c400bb20 con 0x7fc2cc0ad870 2023-11-24T07:58:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.283+0000 7fc2d0f3c700 1 -- 172.21.15.32:0/3399552975 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4012430 con 0x7fc2cc0ad870 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 -- 172.21.15.32:0/3399552975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc0ad870 msgr2=0x7fc2cc06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/3399552975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc0ad870 0x7fc2cc06f6d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009d90 tx=0x7fc2c400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 -- 172.21.15.32:0/3399552975 shutdown_connections 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/3399552975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc0ad870 0x7fc2cc06f6d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/3399552975 >> v2:172.21.15.123:3300/0 conn(0x7fc2cc15e290 0x7fc2cc0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 -- 172.21.15.32:0/3399552975 >> 172.21.15.32:0/3399552975 conn(0x7fc2cc0b2050 msgr2=0x7fc2cc0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 -- 172.21.15.32:0/3399552975 shutdown_connections 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.284+0000 7fc2d41a2700 1 -- 172.21.15.32:0/3399552975 wait complete. 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.285+0000 7fc2d41a2700 1 Processor -- start 2023-11-24T07:58:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.285+0000 7fc2d41a2700 1 -- start start 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d41a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 0x7fc2cc14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d41a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc0aedc0 0x7fc2cc0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d41a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc15e290 0x7fc2cc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d41a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0ce6f0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d41a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0ce9d0 con 0x7fc2cc0aedc0 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d41a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0cecb0 con 0x7fc2cc15e290 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 0x7fc2cc14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 0x7fc2cc14f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56948/0 (socket says 172.21.15.32:56948) 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 -- 172.21.15.32:0/1561527225 learned_addr learned my addr 172.21.15.32:0/1561527225 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d173d700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc0aedc0 0x7fc2cc0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d273f700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc15e290 0x7fc2cc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 -- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc15e290 msgr2=0x7fc2cc152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc15e290 0x7fc2cc152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 -- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc0aedc0 msgr2=0x7fc2cc0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc0aedc0 0x7fc2cc0cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 -- 172.21.15.32:0/1561527225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80094a0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.286+0000 7fc2d1f3e700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 0x7fc2cc14f130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c700 tx=0x7fc2bc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.287+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc005d70 con 0x7fc2cc0ad870 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.287+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc0070d0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.287+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c4009a70 con 0x7fc2cc0ad870 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.287+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc007460 con 0x7fc2cc0ad870 2023-11-24T07:58:49.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.287+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc152050 con 0x7fc2cc0ad870 2023-11-24T07:58:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.288+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2cc068d90 con 0x7fc2cc0ad870 2023-11-24T07:58:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.288+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2bc0075c0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.289+0000 7fc2c27fc700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc2b006aed0 0x7fc2b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.289+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2bc088a80 con 0x7fc2cc0ad870 2023-11-24T07:58:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.289+0000 7fc2d173d700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc2b006aed0 0x7fc2b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.289+0000 7fc2d173d700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc2b006aed0 0x7fc2b006d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc161480 tx=0x7fc2c8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.293+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2bc053a90 con 0x7fc2cc0ad870 2023-11-24T07:58:49.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.495+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2cc068fb0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.495+0000 7fc2c27fc700 1 -- 172.21.15.32:0/1561527225 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2bc0170a0 con 0x7fc2cc0ad870 2023-11-24T07:58:49.498 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:58:49.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc2b006aed0 msgr2=0x7fc2b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc2b006aed0 0x7fc2b006d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc161480 tx=0x7fc2c8009210 comp rx=0 tx=0).stop 2023-11-24T07:58:49.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 msgr2=0x7fc2cc14f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 0x7fc2cc14f130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c700 tx=0x7fc2bc00ca10 comp rx=0 tx=0).stop 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 shutdown_connections 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc2b006aed0 0x7fc2b006d390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc2cc15e290 0x7fc2cc152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2cc0ad870 0x7fc2cc14f130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 --2- 172.21.15.32:0/1561527225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2cc0aedc0 0x7fc2cc0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.497+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 >> 172.21.15.32:0/1561527225 conn(0x7fc2cc0b2050 msgr2=0x7fc2cc0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.498+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 shutdown_connections 2023-11-24T07:58:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.498+0000 7fc2d41a2700 1 -- 172.21.15.32:0/1561527225 wait complete. 2023-11-24T07:58:49.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.638+0000 7fa3e22f1700 1 Processor -- start 2023-11-24T07:58:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.638+0000 7fa3e22f1700 1 -- start start 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3e22f1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa3dc15e150 0x7fa3dc06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3e22f1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0b0590 con 0x7fa3dc0b3bf0 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3e22f1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc06ecd0 con 0x7fa3dc073f90 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3e22f1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b4360 con 0x7fa3dc15e150 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3db7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3dc0b3bf0 0x7fa3dc06e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58668/0 (socket says 172.21.15.32:58668) 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3db7fe700 1 -- 172.21.15.32:0/4290861479 learned_addr learned my addr 172.21.15.32:0/4290861479 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3dbfff700 1 --2- 172.21.15.32:0/4290861479 >> v2:172.21.15.123:3300/0 conn(0x7fa3dc15e150 0x7fa3dc06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649262484 0 0) 0x7fa3dc0b0590 con 0x7fa3dc0b3bf0 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c40035b0 con 0x7fa3dc0b3bf0 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2308716921 0 0) 0x7fa3c40035b0 con 0x7fa3dc0b3bf0 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 >> v2:172.21.15.123:3300/0 conn(0x7fa3dc15e150 msgr2=0x7fa3dc06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 --2- 172.21.15.32:0/4290861479 >> v2:172.21.15.123:3300/0 conn(0x7fa3dc15e150 0x7fa3dc06f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 >> v1:172.21.15.123:6789/0 conn(0x7fa3dc073f90 legacy=0x7fa3dc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0b4080 con 0x7fa3dc0b3bf0 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fa3d0001670 con 0x7fa3dc0b3bf0 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 >> v1:172.21.15.32:6789/0 conn(0x7fa3dc0b3bf0 legacy=0x7fa3dc06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.639+0000 7fa3daffd700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3c4003a00 0x7fa3c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c4005210 0x7fa3c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c400c080 con 0x7fa3c4004850 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c400c1f0 con 0x7fa3c4005210 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c400c3b0 con 0x7fa3c4003a00 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3dbfff700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3c4003a00 0x7fa3c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3e088e700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c4005210 0x7fa3c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 -- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3c4003a00 msgr2=0x7fa3c4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3c4003a00 0x7fa3c4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 -- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c4005210 msgr2=0x7fa3c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c4005210 0x7fa3c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3e088e700 1 -- 172.21.15.32:0/4290861479 reap_dead start 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 -- 172.21.15.32:0/4290861479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40035b0 con 0x7fa3c4004850 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3dbfff700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3c4003a00 0x7fa3c4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3c4004cd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c240 tx=0x7fa3c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc011304251de39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0007c10 con 0x7fa3c4004850 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d000ce00 con 0x7fa3c4004850 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.640+0000 7fa3daffd700 1 -- 172.21.15.32:0/4290861479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0005ba0 con 0x7fa3c4004850 2023-11-24T07:58:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4290861479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 msgr2=0x7fa3c4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3c4004cd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c240 tx=0x7fa3c4005880 comp rx=0 tx=0).stop 2023-11-24T07:58:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4290861479 shutdown_connections 2023-11-24T07:58:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4290861479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3c4004cd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4290861479 >> 172.21.15.32:0/4290861479 conn(0x7fa3dc0d0be0 msgr2=0x7fa3dc0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4290861479 shutdown_connections 2023-11-24T07:58:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.641+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4290861479 wait complete. 2023-11-24T07:58:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.642+0000 7fa3e22f1700 1 Processor -- start 2023-11-24T07:58:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.642+0000 7fa3e22f1700 1 -- start start 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e22f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3dc155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e22f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3dc1f4a30 0x7fa3dc1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e22f1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3dc1f3920 0x7fa3dc0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e22f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1f6c50 con 0x7fa3c4004850 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e22f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc15fea0 con 0x7fa3dc1f4a30 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e22f1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1f4100 con 0x7fa3dc1f3920 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3dc155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3dc155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56966/0 (socket says 172.21.15.32:56966) 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 -- 172.21.15.32:0/4294907108 learned_addr learned my addr 172.21.15.32:0/4294907108 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3dc1f4a30 0x7fa3dc1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3e088e700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3dc1f3920 0x7fa3dc0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 -- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3dc1f3920 msgr2=0x7fa3dc0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3dc1f3920 0x7fa3dc0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 -- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3dc1f4a30 msgr2=0x7fa3dc1f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3dc1f4a30 0x7fa3dc1f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 -- 172.21.15.32:0/4294907108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc1f43e0 con 0x7fa3c4004850 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3dc1f4a30 0x7fa3dc1f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.643+0000 7fa3dbfff700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3dc155d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc011bd0 tx=0x7fa3cc011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.644+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00fa00 con 0x7fa3c4004850 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.644+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00fb60 con 0x7fa3c4004850 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.644+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc1f46c0 con 0x7fa3c4004850 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.644+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004020 con 0x7fa3c4004850 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.644+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc1f3d70 con 0x7fa3c4004850 2023-11-24T07:58:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.645+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3cc004180 con 0x7fa3c4004850 2023-11-24T07:58:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.645+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc068fb0 con 0x7fa3c4004850 2023-11-24T07:58:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.646+0000 7fa3d8ff9700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa3c806aed0 0x7fa3c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.646+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3cc089a00 con 0x7fa3c4004850 2023-11-24T07:58:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.646+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa3c806aed0 0x7fa3c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.646+0000 7fa3db7fe700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa3c806aed0 0x7fa3c806d390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0001930 tx=0x7fa3d0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.650+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3cc054ab0 con 0x7fa3c4004850 2023-11-24T07:58:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.851+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3dc153b00 con 0x7fa3c4004850 2023-11-24T07:58:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.851+0000 7fa3d8ff9700 1 -- 172.21.15.32:0/4294907108 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3cc022070 con 0x7fa3c4004850 2023-11-24T07:58:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:58:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa3c806aed0 msgr2=0x7fa3c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa3c806aed0 0x7fa3c806d390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0001930 tx=0x7fa3d0007000 comp rx=0 tx=0).stop 2023-11-24T07:58:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 msgr2=0x7fa3dc155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3dc155d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc011bd0 tx=0x7fa3cc011f90 comp rx=0 tx=0).stop 2023-11-24T07:58:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 shutdown_connections 2023-11-24T07:58:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa3c806aed0 0x7fa3c806d390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3dc1f3920 0x7fa3dc0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c4004850 0x7fa3dc155d10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 --2- 172.21.15.32:0/4294907108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3dc1f4a30 0x7fa3dc1f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 >> 172.21.15.32:0/4294907108 conn(0x7fa3dc0d0be0 msgr2=0x7fa3dc0af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 shutdown_connections 2023-11-24T07:58:49.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.853+0000 7fa3e22f1700 1 -- 172.21.15.32:0/4294907108 wait complete. 2023-11-24T07:58:49.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.990+0000 7f71d6922700 1 Processor -- start 2023-11-24T07:58:49.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.990+0000 7f71d6922700 1 -- start start 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d6922700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71d01446a0 0x7f71d00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d6922700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d6922700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d00af950 con 0x7f71d006c540 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d6922700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d0144a80 con 0x7f71d01446a0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d6922700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d006c9b0 con 0x7f71d00adf00 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d4ebf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71d006c540 0x7f71d006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58690/0 (socket says 172.21.15.32:58690) 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71d4ebf700 1 -- 172.21.15.32:0/144861436 learned_addr learned my addr 172.21.15.32:0/144861436 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71cf7fe700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71cffff700 1 --2- 172.21.15.32:0/144861436 >> v2:172.21.15.123:3300/0 conn(0x7f71d01446a0 0x7f71d00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235376910 0 0) 0x7f71d00af950 con 0x7f71d006c540 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71bc0035b0 con 0x7f71d006c540 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913442534 0 0) 0x7f71bc0035b0 con 0x7f71d006c540 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 msgr2=0x7f71d006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 >> v2:172.21.15.123:3300/0 conn(0x7f71d01446a0 msgr2=0x7f71d00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 --2- 172.21.15.32:0/144861436 >> v2:172.21.15.123:3300/0 conn(0x7f71d01446a0 0x7f71d00b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d00afc30 con 0x7f71d006c540 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.991+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f71c80019f0 con 0x7f71d006c540 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 >> v1:172.21.15.32:6789/0 conn(0x7f71d006c540 legacy=0x7f71d006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71bc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc003fb0 0x7f71bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71bc004d90 0x7f71bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d0144a80 con 0x7f71bc004d90 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d006c9b0 con 0x7f71bc003fb0 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc00ba10 con 0x7f71d00adf00 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71bc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cf7fe700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc003fb0 0x7f71bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71d4ebf700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71bc004d90 0x7f71bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 -- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc003fb0 msgr2=0x7f71bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc003fb0 0x7f71bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 -- 172.21.15.32:0/144861436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71bc004d90 msgr2=0x7f71bc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71bc004d90 0x7f71bc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 -- 172.21.15.32:0/144861436 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc0035b0 con 0x7f71d00adf00 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cf7fe700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc003fb0 0x7f71bc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71d4ebf700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71bc004d90 0x7f71bc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71cffff700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71bc003a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009b60 tx=0x7f71bc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6758d7b656fac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000e6c0 con 0x7f71d00adf00 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c000e820 con 0x7f71d00adf00 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.992+0000 7f71ceffd700 1 -- 172.21.15.32:0/144861436 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000eb90 con 0x7f71d00adf00 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d6922700 1 -- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 msgr2=0x7f71bc003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d6922700 1 --2- 172.21.15.32:0/144861436 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71bc003a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009b60 tx=0x7f71bc005880 comp rx=0 tx=0).stop 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d4ebf700 1 -- 172.21.15.32:0/144861436 reap_dead start 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d6922700 1 -- 172.21.15.32:0/144861436 shutdown_connections 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d6922700 1 -- 172.21.15.32:0/144861436 >> 172.21.15.32:0/144861436 conn(0x7f71d0071b90 msgr2=0x7f71d0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d6922700 1 -- 172.21.15.32:0/144861436 shutdown_connections 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.993+0000 7f71d6922700 1 -- 172.21.15.32:0/144861436 wait complete. 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.994+0000 7f71d6922700 1 Processor -- start 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.994+0000 7f71d6922700 1 -- start start 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d6922700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d6922700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 0x7f71d014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d6922700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71d01492e0 0x7f71d0149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d6922700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d014b0b0 con 0x7f71d00cffc0 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d6922700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d014a010 con 0x7f71d01492e0 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d6922700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d014a2f0 con 0x7f71d00adf00 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 0x7f71d014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 0x7f71d014b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56988/0 (socket says 172.21.15.32:56988) 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 -- 172.21.15.32:0/2051341031 learned_addr learned my addr 172.21.15.32:0/2051341031 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cffff700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d4ebf700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71d01492e0 0x7f71d0149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 -- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 msgr2=0x7f71d014b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d014b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 -- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71d01492e0 msgr2=0x7f71d0149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71d01492e0 0x7f71d0149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71cf7fe700 1 -- 172.21.15.32:0/2051341031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d014a5d0 con 0x7f71d00cffc0 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.995+0000 7f71d4ebf700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71d01492e0 0x7f71d0149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.996+0000 7f71cf7fe700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 0x7f71d014b8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f71b8011cb0 tx=0x7f71b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.996+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800fa00 con 0x7f71d00cffc0 2023-11-24T07:58:50.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.996+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b800fb60 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.996+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d014a8b0 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.996+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800fed0 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.996+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d0149cb0 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.997+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d00753f0 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.997+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f71b8014440 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.998+0000 7f71ccff9700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f71b006aed0 0x7f71b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.998+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71b8089e20 con 0x7f71d00cffc0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.998+0000 7f71cffff700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f71b006aed0 0x7f71b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:49.998+0000 7f71cffff700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f71b006aed0 0x7f71b006d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71c00090b0 tx=0x7f71c001f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.002+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71b8054ed0 con 0x7f71d00cffc0 2023-11-24T07:58:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.249+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71d0067f60 con 0x7f71d00cffc0 2023-11-24T07:58:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.249+0000 7f71ccff9700 1 -- 172.21.15.32:0/2051341031 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f71b8022020 con 0x7f71d00cffc0 2023-11-24T07:58:50.251 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:50.251 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:58:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.251+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f71b006aed0 msgr2=0x7f71b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.251+0000 7f71d6922700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f71b006aed0 0x7f71b006d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71c00090b0 tx=0x7f71c001f040 comp rx=0 tx=0).stop 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.251+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 msgr2=0x7f71d014b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.251+0000 7f71d6922700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 0x7f71d014b8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f71b8011cb0 tx=0x7f71b8009e30 comp rx=0 tx=0).stop 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.251+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 shutdown_connections 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f71b006aed0 0x7f71b006d390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71d01492e0 0x7f71d0149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71d00adf00 0x7f71d014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 --2- 172.21.15.32:0/2051341031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d00cffc0 0x7f71d014b8b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 >> 172.21.15.32:0/2051341031 conn(0x7f71d0071b90 msgr2=0x7f71d00b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 shutdown_connections 2023-11-24T07:58:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.252+0000 7f71d6922700 1 -- 172.21.15.32:0/2051341031 wait complete. 2023-11-24T07:58:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:58:50.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.388+0000 7f658a358700 1 Processor -- start 2023-11-24T07:58:50.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.389+0000 7f658a358700 1 -- start start 2023-11-24T07:58:50.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.389+0000 7f658a358700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 0x7f65840b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.389+0000 7f658a358700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65840ae430 con 0x7f65840af4d0 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.389+0000 7f658a358700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65840ae710 con 0x7f658415e080 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.389+0000 7f658a358700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65840af0b0 con 0x7f658406c710 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f65888f5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65840af4d0 0x7f658406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58710/0 (socket says 172.21.15.32:58710) 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f65888f5700 1 -- 172.21.15.32:0/3003948455 learned_addr learned my addr 172.21.15.32:0/3003948455 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f65837fe700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 0x7f65840b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840518521 0 0) 0x7f65840ae430 con 0x7f65840af4d0 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f656c0035b0 con 0x7f65840af4d0 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189706123 0 0) 0x7f656c0035b0 con 0x7f65840af4d0 2023-11-24T07:58:50.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 msgr2=0x7f65840b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 0x7f65840b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 >> v1:172.21.15.123:6789/0 conn(0x7f658415e080 legacy=0x7f658406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65840ae9f0 con 0x7f65840af4d0 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f657c0019f0 con 0x7f65840af4d0 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 >> v1:172.21.15.32:6789/0 conn(0x7f65840af4d0 legacy=0x7f658406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f656c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 0x7f656c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c004d90 0x7f656c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65840af0b0 con 0x7f656c003a30 2023-11-24T07:58:50.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c00bac0 con 0x7f656c004d90 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c00bc80 con 0x7f658406c710 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f6583fff700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f656c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f65888f5700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c004d90 0x7f656c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.390+0000 7f65837fe700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 0x7f656c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6583fff700 1 -- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 msgr2=0x7f656c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6583fff700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658406c710 0x7f656c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6583fff700 1 -- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c004d90 msgr2=0x7f656c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6583fff700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f656c004d90 0x7f656c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6583fff700 1 -- 172.21.15.32:0/3003948455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c0035b0 con 0x7f656c003a30 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6583fff700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f656c003e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6574007fd0 tx=0x7f656c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eefe1c6b1fbea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740052d0 con 0x7f656c003a30 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6574005470 con 0x7f656c003a30 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f6582ffd700 1 -- 172.21.15.32:0/3003948455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740038c0 con 0x7f656c003a30 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f658a358700 1 -- 172.21.15.32:0/3003948455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 msgr2=0x7f656c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f658a358700 1 --2- 172.21.15.32:0/3003948455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f656c003e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6574007fd0 tx=0x7f656c005880 comp rx=0 tx=0).stop 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f65888f5700 1 -- 172.21.15.32:0/3003948455 reap_dead start 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f658a358700 1 -- 172.21.15.32:0/3003948455 shutdown_connections 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f658a358700 1 -- 172.21.15.32:0/3003948455 >> 172.21.15.32:0/3003948455 conn(0x7f6584071b90 msgr2=0x7f6584071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f658a358700 1 -- 172.21.15.32:0/3003948455 shutdown_connections 2023-11-24T07:58:50.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.391+0000 7f658a358700 1 -- 172.21.15.32:0/3003948455 wait complete. 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.393+0000 7f658a358700 1 Processor -- start 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.393+0000 7f658a358700 1 -- start start 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.393+0000 7f658a358700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f65841f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584157950 0x7f65840cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840cebb0 0x7f65840cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584159b00 con 0x7f656c003a30 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584158ef0 con 0x7f6584157950 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65841f4930 con 0x7f65840cebb0 2023-11-24T07:58:50.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f65841f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f65841f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57006/0 (socket says 172.21.15.32:57006) 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 -- 172.21.15.32:0/2427265133 learned_addr learned my addr 172.21.15.32:0/2427265133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f65888f5700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840cebb0 0x7f65840cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f65837fe700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584157950 0x7f65840cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 -- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840cebb0 msgr2=0x7f65840cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840cebb0 0x7f65840cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 -- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584157950 msgr2=0x7f65840cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584157950 0x7f65840cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 -- 172.21.15.32:0/2427265133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584157e50 con 0x7f656c003a30 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6583fff700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f65841f43f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6574004000 tx=0x7f6574001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574007240 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65740073a0 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6584158130 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574001290 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.394+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65841586f0 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.395+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65840753f0 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.396+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6574003680 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.396+0000 7f6580ff9700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f657006aed0 0x7f657006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.396+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f657408a170 con 0x7f656c003a30 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.396+0000 7f65837fe700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f657006aed0 0x7f657006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.397+0000 7f65837fe700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f657006aed0 0x7f657006d390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65840d05f0 tx=0x7f6578008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.401+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65740553d0 con 0x7f656c003a30 2023-11-24T07:58:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.602+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6584067f60 con 0x7f656c003a30 2023-11-24T07:58:50.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.603+0000 7f6580ff9700 1 -- 172.21.15.32:0/2427265133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f65740589f0 con 0x7f656c003a30 2023-11-24T07:58:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:50.605 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:50.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.605+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f657006aed0 msgr2=0x7f657006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.605+0000 7f658a358700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f657006aed0 0x7f657006d390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65840d05f0 tx=0x7f6578008040 comp rx=0 tx=0).stop 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.605+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 msgr2=0x7f65841f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.605+0000 7f658a358700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f65841f43f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6574004000 tx=0x7f6574001120 comp rx=0 tx=0).stop 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 shutdown_connections 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f657006aed0 0x7f657006d390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65840cebb0 0x7f65840cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656c003a30 0x7f65841f43f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 --2- 172.21.15.32:0/2427265133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6584157950 0x7f65840cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 >> 172.21.15.32:0/2427265133 conn(0x7f6584071b90 msgr2=0x7f658415d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 shutdown_connections 2023-11-24T07:58:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.606+0000 7f658a358700 1 -- 172.21.15.32:0/2427265133 wait complete. 2023-11-24T07:58:50.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:50.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.747+0000 7fd35f1c0700 1 Processor -- start 2023-11-24T07:58:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.747+0000 7fd35f1c0700 1 -- start start 2023-11-24T07:58:50.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35f1c0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3581587f0 0x7fd35806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35f1c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd35815b370 0x7fd35815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35f1c0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd3580b3c60 0x7fd3580b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35f1c0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3580b4610 con 0x7fd3581587f0 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35f1c0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35806b530 con 0x7fd3580b3c60 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35f1c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580ae5e0 con 0x7fd35815b370 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3581587f0 0x7fd35806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3581587f0 0x7fd35806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57014/0 (socket says 172.21.15.32:57014) 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 -- 172.21.15.32:0/2459559956 learned_addr learned my addr 172.21.15.32:0/2459559956 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35d75d700 1 --2- 172.21.15.32:0/2459559956 >> v2:172.21.15.123:3300/0 conn(0x7fd3580b3c60 0x7fd3580b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd357fff700 1 --2- 172.21.15.32:0/2459559956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd35815b370 0x7fd35815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 -- 172.21.15.32:0/2459559956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd35815b370 msgr2=0x7fd35815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:50.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 --2- 172.21.15.32:0/2459559956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd35815b370 0x7fd35815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 -- 172.21.15.32:0/2459559956 >> v2:172.21.15.123:3300/0 conn(0x7fd3580b3c60 msgr2=0x7fd3580b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 --2- 172.21.15.32:0/2459559956 >> v2:172.21.15.123:3300/0 conn(0x7fd3580b3c60 0x7fd3580b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 -- 172.21.15.32:0/2459559956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35815be30 con 0x7fd3581587f0 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35d75d700 1 --2- 172.21.15.32:0/2459559956 >> v2:172.21.15.123:3300/0 conn(0x7fd3580b3c60 0x7fd3580b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.748+0000 7fd35cf5c700 1 --2- 172.21.15.32:0/2459559956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3581587f0 0x7fd35806aff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd35815bd10 tx=0x7fd34c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e3628f112c6f05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd3577fe700 1 -- 172.21.15.32:0/2459559956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c011830 con 0x7fd3581587f0 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd3577fe700 1 -- 172.21.15.32:0/2459559956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c011990 con 0x7fd3581587f0 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd3577fe700 1 -- 172.21.15.32:0/2459559956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c011d20 con 0x7fd3581587f0 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2459559956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3581587f0 msgr2=0x7fd35806aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2459559956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3581587f0 0x7fd35806aff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd35815bd10 tx=0x7fd34c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2459559956 shutdown_connections 2023-11-24T07:58:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2459559956 >> v2:172.21.15.123:3300/0 conn(0x7fd3580b3c60 0x7fd3580b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2459559956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3581587f0 0x7fd35806aff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2459559956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd35815b370 0x7fd35815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2459559956 >> 172.21.15.32:0/2459559956 conn(0x7fd3580d0bf0 msgr2=0x7fd3580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2459559956 shutdown_connections 2023-11-24T07:58:50.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.749+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2459559956 wait complete. 2023-11-24T07:58:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.751+0000 7fd35f1c0700 1 Processor -- start 2023-11-24T07:58:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.751+0000 7fd35f1c0700 1 -- start start 2023-11-24T07:58:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35f1c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580b3c60 0x7fd3581f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35f1c0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3581587f0 0x7fd3580c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35f1c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 0x7fd35815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35f1c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c69d0 con 0x7fd35815b370 2023-11-24T07:58:50.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35f1c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c6420 con 0x7fd3580b3c60 2023-11-24T07:58:50.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35f1c0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c66f0 con 0x7fd3581587f0 2023-11-24T07:58:50.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 0x7fd35815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 0x7fd35815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57022/0 (socket says 172.21.15.32:57022) 2023-11-24T07:58:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 -- 172.21.15.32:0/2182289902 learned_addr learned my addr 172.21.15.32:0/2182289902 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35cf5c700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580b3c60 0x7fd3581f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd357fff700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3581587f0 0x7fd3580c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 -- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3581587f0 msgr2=0x7fd3580c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3581587f0 0x7fd3580c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 -- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580b3c60 msgr2=0x7fd3581f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580b3c60 0x7fd3581f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.752+0000 7fd35d75d700 1 -- 172.21.15.32:0/2182289902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009550 con 0x7fd35815b370 2023-11-24T07:58:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.753+0000 7fd35d75d700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 0x7fd35815d8a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd350007ef0 tx=0x7fd35000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.753+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000dd80 con 0x7fd35815b370 2023-11-24T07:58:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.753+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000f040 con 0x7fd35815b370 2023-11-24T07:58:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.753+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3581f1980 con 0x7fd35815b370 2023-11-24T07:58:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.753+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350003c60 con 0x7fd35815b370 2023-11-24T07:58:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.753+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3581f1f40 con 0x7fd35815b370 2023-11-24T07:58:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.754+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3581f17e0 con 0x7fd35815b370 2023-11-24T07:58:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.754+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd350004030 con 0x7fd35815b370 2023-11-24T07:58:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.754+0000 7fd3557fa700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd33c06aed0 0x7fd33c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.755+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd350089690 con 0x7fd35815b370 2023-11-24T07:58:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.755+0000 7fd35cf5c700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd33c06aed0 0x7fd33c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.755+0000 7fd35cf5c700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd33c06aed0 0x7fd33c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009c10 tx=0x7fd34c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.759+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3581f17e0 con 0x7fd35815b370 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.958+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3581f17e0 con 0x7fd35815b370 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.958+0000 7fd3557fa700 1 -- 172.21.15.32:0/2182289902 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3581f17e0 con 0x7fd35815b370 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd33c06aed0 msgr2=0x7fd33c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd33c06aed0 0x7fd33c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009c10 tx=0x7fd34c008030 comp rx=0 tx=0).stop 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 msgr2=0x7fd35815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 0x7fd35815d8a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd350007ef0 tx=0x7fd35000d3b0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 shutdown_connections 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd33c06aed0 0x7fd33c06d390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35815b370 0x7fd35815d8a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3580b3c60 0x7fd3581f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 --2- 172.21.15.32:0/2182289902 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd3581587f0 0x7fd3580c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 >> 172.21.15.32:0/2182289902 conn(0x7fd3580d0bf0 msgr2=0x7fd35815a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 shutdown_connections 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:50.960+0000 7fd35f1c0700 1 -- 172.21.15.32:0/2182289902 wait complete. 2023-11-24T07:58:50.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.100+0000 7fd7933cf700 1 Processor -- start 2023-11-24T07:58:51.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.100+0000 7fd7933cf700 1 -- start start 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd7933cf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd78c15e160 0x7fd78c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd7933cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd78c0afc90 0x7fd78c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd7933cf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c0b3c60 con 0x7fd78c0744d0 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd7933cf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c0b4500 con 0x7fd78c0afc90 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd7933cf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c074940 con 0x7fd78c15e160 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79096a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd78c0afc90 0x7fd78c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79196c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd78c0744d0 0x7fd78c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58742/0 (socket says 172.21.15.32:58742) 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79196c700 1 -- 172.21.15.32:0/4032448168 learned_addr learned my addr 172.21.15.32:0/4032448168 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79116b700 1 --2- 172.21.15.32:0/4032448168 >> v2:172.21.15.123:3300/0 conn(0x7fd78c15e160 0x7fd78c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79096a700 1 -- 172.21.15.32:0/4032448168 >> v2:172.21.15.123:3300/0 conn(0x7fd78c15e160 msgr2=0x7fd78c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79096a700 1 --2- 172.21.15.32:0/4032448168 >> v2:172.21.15.123:3300/0 conn(0x7fd78c15e160 0x7fd78c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79096a700 1 -- 172.21.15.32:0/4032448168 >> v1:172.21.15.32:6789/0 conn(0x7fd78c0744d0 legacy=0x7fd78c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79096a700 1 -- 172.21.15.32:0/4032448168 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0b58c0 con 0x7fd78c0afc90 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79116b700 1 --2- 172.21.15.32:0/4032448168 >> v2:172.21.15.123:3300/0 conn(0x7fd78c15e160 0x7fd78c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.101+0000 7fd79096a700 1 --2- 172.21.15.32:0/4032448168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c0afc90 0x7fd78c073f90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd78c09efa0 tx=0x7fd788009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4556901056bf2258 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd783fff700 1 -- 172.21.15.32:0/4032448168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011830 con 0x7fd78c0afc90 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd783fff700 1 -- 172.21.15.32:0/4032448168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788011990 con 0x7fd78c0afc90 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd783fff700 1 -- 172.21.15.32:0/4032448168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011d20 con 0x7fd78c0afc90 2023-11-24T07:58:51.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 -- 172.21.15.32:0/4032448168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c0afc90 msgr2=0x7fd78c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 --2- 172.21.15.32:0/4032448168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c0afc90 0x7fd78c073f90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd78c09efa0 tx=0x7fd788009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 -- 172.21.15.32:0/4032448168 shutdown_connections 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 --2- 172.21.15.32:0/4032448168 >> v2:172.21.15.123:3300/0 conn(0x7fd78c15e160 0x7fd78c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 --2- 172.21.15.32:0/4032448168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c0afc90 0x7fd78c073f90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 -- 172.21.15.32:0/4032448168 >> 172.21.15.32:0/4032448168 conn(0x7fd78c0d0bf0 msgr2=0x7fd78c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 -- 172.21.15.32:0/4032448168 shutdown_connections 2023-11-24T07:58:51.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.102+0000 7fd7933cf700 1 -- 172.21.15.32:0/4032448168 wait complete. 2023-11-24T07:58:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 Processor -- start 2023-11-24T07:58:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 -- start start 2023-11-24T07:58:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd78c0744d0 0x7fd78c149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd78c0afc90 0x7fd78c1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 0x7fd78c1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0c8fc0 con 0x7fd78c15e160 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0d0130 con 0x7fd78c0744d0 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd7933cf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c160340 con 0x7fd78c0afc90 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd79196c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 0x7fd78c1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd79196c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 0x7fd78c1496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57036/0 (socket says 172.21.15.32:57036) 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.104+0000 7fd79196c700 1 -- 172.21.15.32:0/2467775709 learned_addr learned my addr 172.21.15.32:0/2467775709 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79116b700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd78c0744d0 0x7fd78c149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79096a700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd78c0afc90 0x7fd78c1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79196c700 1 -- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd78c0afc90 msgr2=0x7fd78c1f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79196c700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd78c0afc90 0x7fd78c1f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79196c700 1 -- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd78c0744d0 msgr2=0x7fd78c149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79196c700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd78c0744d0 0x7fd78c149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79196c700 1 -- 172.21.15.32:0/2467775709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880094a0 con 0x7fd78c15e160 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79096a700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd78c0afc90 0x7fd78c1f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79116b700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd78c0744d0 0x7fd78c149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd79196c700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 0x7fd78c1496b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ca80 tx=0x7fd78400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400aaa0 con 0x7fd78c15e160 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400ac00 con 0x7fd78c15e160 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c1f81d0 con 0x7fd78c15e160 2023-11-24T07:58:51.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784012400 con 0x7fd78c15e160 2023-11-24T07:58:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.105+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c15fa10 con 0x7fd78c15e160 2023-11-24T07:58:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.106+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c068fb0 con 0x7fd78c15e160 2023-11-24T07:58:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.107+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd78401b3b0 con 0x7fd78c15e160 2023-11-24T07:58:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.107+0000 7fd781ffb700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd77006aed0 0x7fd77006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.107+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7840891e0 con 0x7fd78c15e160 2023-11-24T07:58:51.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.107+0000 7fd79116b700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd77006aed0 0x7fd77006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.108+0000 7fd79116b700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd77006aed0 0x7fd77006d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009a50 tx=0x7fd77c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.111+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7840542c0 con 0x7fd78c15e160 2023-11-24T07:58:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.311+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd78c160480 con 0x7fd78c15e160 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.311+0000 7fd781ffb700 1 -- 172.21.15.32:0/2467775709 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd78c160480 con 0x7fd78c15e160 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd77006aed0 msgr2=0x7fd77006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd77006aed0 0x7fd77006d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009a50 tx=0x7fd77c008040 comp rx=0 tx=0).stop 2023-11-24T07:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 msgr2=0x7fd78c1496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 0x7fd78c1496b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ca80 tx=0x7fd78400ce40 comp rx=0 tx=0).stop 2023-11-24T07:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 shutdown_connections 2023-11-24T07:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd77006aed0 0x7fd77006d390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c15e160 0x7fd78c1496b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.313+0000 7fd7933cf700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd78c0744d0 0x7fd78c149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.314+0000 7fd7933cf700 1 --2- 172.21.15.32:0/2467775709 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd78c0afc90 0x7fd78c1f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.314+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 >> 172.21.15.32:0/2467775709 conn(0x7fd78c0d0bf0 msgr2=0x7fd78c06e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.314+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 shutdown_connections 2023-11-24T07:58:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.314+0000 7fd7933cf700 1 -- 172.21.15.32:0/2467775709 wait complete. 2023-11-24T07:58:51.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.454+0000 7fe52d5fe700 1 Processor -- start 2023-11-24T07:58:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.454+0000 7fe52d5fe700 1 -- start start 2023-11-24T07:58:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe52d5fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe52815e140 0x7fe5280d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe52d5fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5280b98c0 0x7fe5280d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe52d5fe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280b80e0 con 0x7fe5280d39d0 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe52d5fe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52815e520 con 0x7fe52815e140 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe52d5fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280d2930 con 0x7fe5280b98c0 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe526ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe52815e140 0x7fe5280d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe526ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe52815e140 0x7fe5280d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57040/0 (socket says 172.21.15.32:57040) 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.455+0000 7fe526ffd700 1 -- 172.21.15.32:0/3862108075 learned_addr learned my addr 172.21.15.32:0/3862108075 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe5277fe700 1 --2- 172.21.15.32:0/3862108075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5280b98c0 0x7fe5280d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe526ffd700 1 -- 172.21.15.32:0/3862108075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5280b98c0 msgr2=0x7fe5280d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe526ffd700 1 --2- 172.21.15.32:0/3862108075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5280b98c0 0x7fe5280d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe526ffd700 1 -- 172.21.15.32:0/3862108075 >> v1:172.21.15.123:6789/0 conn(0x7fe5280d39d0 legacy=0x7fe5280b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe526ffd700 1 -- 172.21.15.32:0/3862108075 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280d2c10 con 0x7fe52815e140 2023-11-24T07:58:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe526ffd700 1 --2- 172.21.15.32:0/3862108075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe52815e140 0x7fe5280d3ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0098f0 tx=0x7fe51c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f283c3958749fcc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe525ffb700 1 -- 172.21.15.32:0/3862108075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004230 con 0x7fe52815e140 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe525ffb700 1 -- 172.21.15.32:0/3862108075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c004390 con 0x7fe52815e140 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe525ffb700 1 -- 172.21.15.32:0/3862108075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0036a0 con 0x7fe52815e140 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe52d5fe700 1 -- 172.21.15.32:0/3862108075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe52815e140 msgr2=0x7fe5280d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/3862108075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe52815e140 0x7fe5280d3ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0098f0 tx=0x7fe51c009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe52d5fe700 1 -- 172.21.15.32:0/3862108075 shutdown_connections 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/3862108075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5280b98c0 0x7fe5280d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/3862108075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe52815e140 0x7fe5280d3ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.456+0000 7fe52d5fe700 1 -- 172.21.15.32:0/3862108075 >> 172.21.15.32:0/3862108075 conn(0x7fe52806b410 msgr2=0x7fe52806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.457+0000 7fe52d5fe700 1 -- 172.21.15.32:0/3862108075 shutdown_connections 2023-11-24T07:58:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.457+0000 7fe52d5fe700 1 -- 172.21.15.32:0/3862108075 wait complete. 2023-11-24T07:58:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.458+0000 7fe52d5fe700 1 Processor -- start 2023-11-24T07:58:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.458+0000 7fe52d5fe700 1 -- start start 2023-11-24T07:58:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 0x7fe52814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5280d39d0 0x7fe5281609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe52815e140 0x7fe528160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281f6b80 con 0x7fe5280b98c0 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281f5f00 con 0x7fe5280d39d0 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281f61e0 con 0x7fe52815e140 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 0x7fe52814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 0x7fe52814f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57044/0 (socket says 172.21.15.32:57044) 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 -- 172.21.15.32:0/108997705 learned_addr learned my addr 172.21.15.32:0/108997705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe5277fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe52815e140 0x7fe528160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe5267fc700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5280d39d0 0x7fe5281609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 -- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe52815e140 msgr2=0x7fe528160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe52815e140 0x7fe528160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 -- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5280d39d0 msgr2=0x7fe5281609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5280d39d0 0x7fe5281609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 -- 172.21.15.32:0/108997705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c009550 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe526ffd700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 0x7fe52814f510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009130 tx=0x7fe51c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c003ce0 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c003e40 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00b880 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.459+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5281f64c0 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.460+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5281f6fa0 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.460+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5281f6df0 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.461+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe51c00b9e0 con 0x7fe5280b98c0 2023-11-24T07:58:51.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.461+0000 7fe50f7fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe51406aed0 0x7fe51406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.461+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe51c08a9d0 con 0x7fe5280b98c0 2023-11-24T07:58:51.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.461+0000 7fe5267fc700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe51406aed0 0x7fe51406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.462+0000 7fe5267fc700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe51406aed0 0x7fe51406d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b7e40 tx=0x7fe518007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.466+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe51c055a90 con 0x7fe5280b98c0 2023-11-24T07:58:51.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.668+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5280b80e0 con 0x7fe5280b98c0 2023-11-24T07:58:51.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.668+0000 7fe50f7fe700 1 -- 172.21.15.32:0/108997705 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5280b80e0 con 0x7fe5280b98c0 2023-11-24T07:58:51.670 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:58:51.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe51406aed0 msgr2=0x7fe51406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe51406aed0 0x7fe51406d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b7e40 tx=0x7fe518007ec0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 msgr2=0x7fe52814f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 0x7fe52814f510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009130 tx=0x7fe51c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:51.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 shutdown_connections 2023-11-24T07:58:51.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe51406aed0 0x7fe51406d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe52815e140 0x7fe528160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5280b98c0 0x7fe52814f510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 --2- 172.21.15.32:0/108997705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5280d39d0 0x7fe5281609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.670+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 >> 172.21.15.32:0/108997705 conn(0x7fe52806b410 msgr2=0x7fe5280d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.671+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 shutdown_connections 2023-11-24T07:58:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.671+0000 7fe52d5fe700 1 -- 172.21.15.32:0/108997705 wait complete. 2023-11-24T07:58:51.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:51.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.808+0000 7f652544d700 1 Processor -- start 2023-11-24T07:58:51.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.808+0000 7f652544d700 1 -- start start 2023-11-24T07:58:51.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f652544d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 0x7f65200d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f652544d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65200d0f80 0x7f65200b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f652544d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200d20b0 con 0x7f65200d3a00 2023-11-24T07:58:51.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f652544d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65200d3150 con 0x7f65200d0f80 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f652544d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652015e5c0 con 0x7f652015e1e0 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 0x7f65200d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 0x7f65200d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:52304/0 (socket says 172.21.15.32:52304) 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65200d0f80 0x7f65200b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651effd700 1 -- 172.21.15.32:0/1021139797 learned_addr learned my addr 172.21.15.32:0/1021139797 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651e7fc700 1 --1- 172.21.15.32:0/1021139797 >> v1:172.21.15.123:6789/0 conn(0x7f65200d3a00 0x7f65200d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45520/0 (socket says 172.21.15.32:45520) 2023-11-24T07:58:51.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651f7fe700 1 -- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 msgr2=0x7f65200d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651f7fe700 1 --2- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 0x7f65200d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651f7fe700 1 -- 172.21.15.32:0/1021139797 >> v1:172.21.15.123:6789/0 conn(0x7f65200d3a00 legacy=0x7f65200d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651f7fe700 1 -- 172.21.15.32:0/1021139797 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200d2390 con 0x7f65200d0f80 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651effd700 1 --2- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 0x7f65200d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.809+0000 7f651f7fe700 1 --2- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d0f80 0x7f65200b6ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6518009dc0 tx=0x7f651800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449ac58407d0f120 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f651dffb700 1 -- 172.21.15.32:0/1021139797 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800e070 con 0x7f65200d0f80 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f651dffb700 1 -- 172.21.15.32:0/1021139797 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800bae0 con 0x7f65200d0f80 2023-11-24T07:58:51.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f651dffb700 1 -- 172.21.15.32:0/1021139797 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800be90 con 0x7f65200d0f80 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 -- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d0f80 msgr2=0x7f65200b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 --2- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d0f80 0x7f65200b6ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6518009dc0 tx=0x7f651800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 -- 172.21.15.32:0/1021139797 shutdown_connections 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 --2- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d0f80 0x7f65200b6ef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 --2- 172.21.15.32:0/1021139797 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015e1e0 0x7f65200d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 -- 172.21.15.32:0/1021139797 >> 172.21.15.32:0/1021139797 conn(0x7f65200b5b50 msgr2=0x7f65200b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 -- 172.21.15.32:0/1021139797 shutdown_connections 2023-11-24T07:58:51.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.810+0000 7f652544d700 1 -- 172.21.15.32:0/1021139797 wait complete. 2023-11-24T07:58:51.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 Processor -- start 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 -- start start 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200d0f80 0x7f652015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 0x7f65200cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f652015e1e0 0x7f65201f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f3b40 con 0x7f65200d3a00 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f3de0 con 0x7f652015e1e0 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f652544d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f40c0 con 0x7f65200d0f80 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f651e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 0x7f65200cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.812+0000 7f651e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 0x7f65200cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57060/0 (socket says 172.21.15.32:57060) 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 -- 172.21.15.32:0/3832665384 learned_addr learned my addr 172.21.15.32:0/3832665384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651effd700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200d0f80 0x7f652015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651f7fe700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f652015e1e0 0x7f65201f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 -- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200d0f80 msgr2=0x7f652015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200d0f80 0x7f652015bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 -- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f652015e1e0 msgr2=0x7f65201f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f652015e1e0 0x7f65201f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 -- 172.21.15.32:0/3832665384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510009550 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f651e7fc700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 0x7f65200cdab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f651400c520 tx=0x7f651400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514005cb0 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6514005e10 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6518009a70 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140072d0 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.813+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65200aaa00 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.814+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65200aabc0 con 0x7f65200d3a00 2023-11-24T07:58:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.815+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6514007430 con 0x7f65200d3a00 2023-11-24T07:58:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.815+0000 7f65077fe700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f650c06aed0 0x7f650c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.815+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6514088950 con 0x7f65200d3a00 2023-11-24T07:58:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.815+0000 7f651effd700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f650c06aed0 0x7f650c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.816+0000 7f651effd700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f650c06aed0 0x7f650c06d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f652006bd10 tx=0x7f6510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:51.820+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6514053b40 con 0x7f65200d3a00 2023-11-24T07:58:52.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.067+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65200aa840 con 0x7f65200d3a00 2023-11-24T07:58:52.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.067+0000 7f65077fe700 1 -- 172.21.15.32:0/3832665384 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f6514017020 con 0x7f65200d3a00 2023-11-24T07:58:52.069 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:52.069 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:58:52.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f650c06aed0 msgr2=0x7f650c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f650c06aed0 0x7f650c06d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f652006bd10 tx=0x7f6510008040 comp rx=0 tx=0).stop 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 msgr2=0x7f65200cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 0x7f65200cdab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f651400c520 tx=0x7f651400c8e0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 shutdown_connections 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f650c06aed0 0x7f650c06d390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f652015e1e0 0x7f65201f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200d0f80 0x7f652015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 --2- 172.21.15.32:0/3832665384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65200d3a00 0x7f65200cdab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 >> 172.21.15.32:0/3832665384 conn(0x7f65200b5b50 msgr2=0x7f652015d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 shutdown_connections 2023-11-24T07:58:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.070+0000 7f652544d700 1 -- 172.21.15.32:0/3832665384 wait complete. 2023-11-24T07:58:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:58:52.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.206+0000 7f09b8b9d700 1 Processor -- start 2023-11-24T07:58:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.206+0000 7f09b8b9d700 1 -- start start 2023-11-24T07:58:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.207+0000 7f09b8b9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.207+0000 7f09b8b9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b41603c0 0x7f09b40d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.207+0000 7f09b8b9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09b40d1a50 0x7f09b40d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.207+0000 7f09b8b9d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b415c2b0 con 0x7f09b41603c0 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.207+0000 7f09b8b9d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b40d0770 con 0x7f09b40d1a50 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.207+0000 7f09b8b9d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40cd100 con 0x7f09b415bed0 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b41603c0 0x7f09b40d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b41603c0 0x7f09b40d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57068/0 (socket says 172.21.15.32:57068) 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 -- 172.21.15.32:0/1719429335 learned_addr learned my addr 172.21.15.32:0/1719429335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b2d9d700 1 --2- 172.21.15.32:0/1719429335 >> v2:172.21.15.123:3300/0 conn(0x7f09b40d1a50 0x7f09b40d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b259c700 1 --2- 172.21.15.32:0/1719429335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 -- 172.21.15.32:0/1719429335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 msgr2=0x7f09b40b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/1719429335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b40b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 -- 172.21.15.32:0/1719429335 >> v2:172.21.15.123:3300/0 conn(0x7f09b40d1a50 msgr2=0x7f09b40d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/1719429335 >> v2:172.21.15.123:3300/0 conn(0x7f09b40d1a50 0x7f09b40d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 -- 172.21.15.32:0/1719429335 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b40d2420 con 0x7f09b41603c0 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/1719429335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b41603c0 0x7f09b40d0230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09b40cf700 tx=0x7f09a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13fd6ab47504105 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b159a700 1 -- 172.21.15.32:0/1719429335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800b870 con 0x7f09b41603c0 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b159a700 1 -- 172.21.15.32:0/1719429335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800b9d0 con 0x7f09b41603c0 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.208+0000 7f09b159a700 1 -- 172.21.15.32:0/1719429335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800bd60 con 0x7f09b41603c0 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 -- 172.21.15.32:0/1719429335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b41603c0 msgr2=0x7f09b40d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/1719429335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b41603c0 0x7f09b40d0230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09b40cf700 tx=0x7f09a8009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 -- 172.21.15.32:0/1719429335 shutdown_connections 2023-11-24T07:58:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/1719429335 >> v2:172.21.15.123:3300/0 conn(0x7f09b40d1a50 0x7f09b40d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/1719429335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b40b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/1719429335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b41603c0 0x7f09b40d0230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 -- 172.21.15.32:0/1719429335 >> 172.21.15.32:0/1719429335 conn(0x7f09b4073130 msgr2=0x7f09b409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 -- 172.21.15.32:0/1719429335 shutdown_connections 2023-11-24T07:58:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.209+0000 7f09b8b9d700 1 -- 172.21.15.32:0/1719429335 wait complete. 2023-11-24T07:58:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.210+0000 7f09b8b9d700 1 Processor -- start 2023-11-24T07:58:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.211+0000 7f09b8b9d700 1 -- start start 2023-11-24T07:58:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.211+0000 7f09b8b9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 0x7f09b409c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.211+0000 7f09b8b9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b4159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.211+0000 7f09b8b9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09b41603c0 0x7f09b40ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b8b9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40c8d20 con 0x7f09b40d1a50 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b8b9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b409da70 con 0x7f09b41603c0 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b8b9d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b409ce60 con 0x7f09b415bed0 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 0x7f09b409c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 0x7f09b409c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57072/0 (socket says 172.21.15.32:57072) 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 -- 172.21.15.32:0/3001002655 learned_addr learned my addr 172.21.15.32:0/3001002655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b2d9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09b41603c0 0x7f09b40ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b4159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 -- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 msgr2=0x7f09b4159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b4159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 -- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09b41603c0 msgr2=0x7f09b40ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09b41603c0 0x7f09b40ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 -- 172.21.15.32:0/3001002655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009550 con 0x7f09b40d1a50 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b2d9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09b41603c0 0x7f09b40ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b4159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b259c700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 0x7f09b409c880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09a400e810 tx=0x7f09a400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4013610 con 0x7f09b40d1a50 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4013770 con 0x7f09b40d1a50 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b409d140 con 0x7f09b40d1a50 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4013b20 con 0x7f09b40d1a50 2023-11-24T07:58:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.212+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b409d700 con 0x7f09b40d1a50 2023-11-24T07:58:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.213+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b4067f60 con 0x7f09b40d1a50 2023-11-24T07:58:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.214+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09a4013c80 con 0x7f09b40d1a50 2023-11-24T07:58:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.214+0000 7f09a2ffd700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f099406aed0 0x7f099406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.214+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09a40892a0 con 0x7f09b40d1a50 2023-11-24T07:58:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.214+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f099406aed0 0x7f099406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.215+0000 7f09b1d9b700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f099406aed0 0x7f099406d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09a8014040 tx=0x7f09a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.219+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09a4054360 con 0x7f09b40d1a50 2023-11-24T07:58:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.419+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09b40aef00 con 0x7f09b40d1a50 2023-11-24T07:58:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.420+0000 7f09a2ffd700 1 -- 172.21.15.32:0/3001002655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f09a4057980 con 0x7f09b40d1a50 2023-11-24T07:58:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:52.422 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f099406aed0 msgr2=0x7f099406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f099406aed0 0x7f099406d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09a8014040 tx=0x7f09a8008030 comp rx=0 tx=0).stop 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 msgr2=0x7f09b409c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 0x7f09b409c880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09a400e810 tx=0x7f09a400ebd0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 shutdown_connections 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f099406aed0 0x7f099406d390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09b41603c0 0x7f09b40ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b40d1a50 0x7f09b409c880 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 --2- 172.21.15.32:0/3001002655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09b415bed0 0x7f09b4159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 >> 172.21.15.32:0/3001002655 conn(0x7f09b4073130 msgr2=0x7f09b40b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 shutdown_connections 2023-11-24T07:58:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.422+0000 7f09b8b9d700 1 -- 172.21.15.32:0/3001002655 wait complete. 2023-11-24T07:58:52.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:52.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.559+0000 7f7a4779f700 1 Processor -- start 2023-11-24T07:58:52.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.559+0000 7f7a4779f700 1 -- start start 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a4779f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a400adf00 0x7f7a4006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a4779f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a4006c540 0x7f7a400b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a4779f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a400af950 con 0x7f7a401446a0 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a4779f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4006c9f0 con 0x7f7a4006c540 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a4779f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a40144a80 con 0x7f7a400adf00 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a4006c540 0x7f7a400b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a4006c540 0x7f7a400b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57084/0 (socket says 172.21.15.32:57084) 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 -- 172.21.15.32:0/3638684946 learned_addr learned my addr 172.21.15.32:0/3638684946 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a44d3a700 1 --2- 172.21.15.32:0/3638684946 >> v2:172.21.15.123:3300/0 conn(0x7f7a400adf00 0x7f7a4006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 -- 172.21.15.32:0/3638684946 >> v2:172.21.15.123:3300/0 conn(0x7f7a400adf00 msgr2=0x7f7a4006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 --2- 172.21.15.32:0/3638684946 >> v2:172.21.15.123:3300/0 conn(0x7f7a400adf00 0x7f7a4006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 -- 172.21.15.32:0/3638684946 >> v1:172.21.15.123:6789/0 conn(0x7f7a401446a0 legacy=0x7f7a400b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 -- 172.21.15.32:0/3638684946 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400afc30 con 0x7f7a4006c540 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.560+0000 7f7a45d3c700 1 --2- 172.21.15.32:0/3638684946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a4006c540 0x7f7a400b3d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a4009f030 tx=0x7f7a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c8c18c0712ee77 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a37fff700 1 -- 172.21.15.32:0/3638684946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800e070 con 0x7f7a4006c540 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a37fff700 1 -- 172.21.15.32:0/3638684946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a38003a20 con 0x7f7a4006c540 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a37fff700 1 -- 172.21.15.32:0/3638684946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38003dd0 con 0x7f7a4006c540 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 -- 172.21.15.32:0/3638684946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a4006c540 msgr2=0x7f7a400b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 --2- 172.21.15.32:0/3638684946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a4006c540 0x7f7a400b3d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a4009f030 tx=0x7f7a3800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:52.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 -- 172.21.15.32:0/3638684946 shutdown_connections 2023-11-24T07:58:52.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 --2- 172.21.15.32:0/3638684946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a4006c540 0x7f7a400b3d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 --2- 172.21.15.32:0/3638684946 >> v2:172.21.15.123:3300/0 conn(0x7f7a400adf00 0x7f7a4006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 -- 172.21.15.32:0/3638684946 >> 172.21.15.32:0/3638684946 conn(0x7f7a40071b90 msgr2=0x7f7a40071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 -- 172.21.15.32:0/3638684946 shutdown_connections 2023-11-24T07:58:52.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.561+0000 7f7a4779f700 1 -- 172.21.15.32:0/3638684946 wait complete. 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.562+0000 7f7a4779f700 1 Processor -- start 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 -- start start 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4006c540 0x7f7a4015bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 0x7f7a40147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a401446a0 0x7f7a400cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4013c1e0 con 0x7f7a400adf00 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4013c4c0 con 0x7f7a4006c540 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a4779f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4013c7a0 con 0x7f7a401446a0 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a44d3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 0x7f7a40147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a44d3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 0x7f7a40147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57088/0 (socket says 172.21.15.32:57088) 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.563+0000 7f7a44d3a700 1 -- 172.21.15.32:0/1405457715 learned_addr learned my addr 172.21.15.32:0/1405457715 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a45d3c700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a401446a0 0x7f7a400cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a4553b700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4006c540 0x7f7a4015bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a44d3a700 1 -- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a401446a0 msgr2=0x7f7a400cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a44d3a700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a401446a0 0x7f7a400cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a44d3a700 1 -- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4006c540 msgr2=0x7f7a4015bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a44d3a700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4006c540 0x7f7a4015bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a44d3a700 1 -- 172.21.15.32:0/1405457715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38009b30 con 0x7f7a400adf00 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a4553b700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4006c540 0x7f7a4015bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a45d3c700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a401446a0 0x7f7a400cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a44d3a700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 0x7f7a40147150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00e7e0 tx=0x7f7a3c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c013610 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c013770 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4013ca80 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c013b20 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.564+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4015c0b0 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.565+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a400753f0 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.566+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a3c013c80 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.566+0000 7f7a35ffb700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a2406aed0 0x7f7a2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.566+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a3c0891f0 con 0x7f7a400adf00 2023-11-24T07:58:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.566+0000 7f7a4553b700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a2406aed0 0x7f7a2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.567+0000 7f7a4553b700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a2406aed0 0x7f7a2406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009930 tx=0x7f7a30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.571+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a3c0542a0 con 0x7f7a400adf00 2023-11-24T07:58:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.815+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a40067f60 con 0x7f7a400adf00 2023-11-24T07:58:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.816+0000 7f7a35ffb700 1 -- 172.21.15.32:0/1405457715 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f7a3c0578c0 con 0x7f7a400adf00 2023-11-24T07:58:52.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:58:52.818 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:58:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a2406aed0 msgr2=0x7f7a2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a2406aed0 0x7f7a2406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009930 tx=0x7f7a30008040 comp rx=0 tx=0).stop 2023-11-24T07:58:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 msgr2=0x7f7a40147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 0x7f7a40147150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00e7e0 tx=0x7f7a3c00eba0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 shutdown_connections 2023-11-24T07:58:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a2406aed0 0x7f7a2406d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a401446a0 0x7f7a400cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4006c540 0x7f7a4015bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 --2- 172.21.15.32:0/1405457715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a400adf00 0x7f7a40147150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 >> 172.21.15.32:0/1405457715 conn(0x7f7a40071b90 msgr2=0x7f7a400b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 shutdown_connections 2023-11-24T07:58:52.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:52.818+0000 7f7a4779f700 1 -- 172.21.15.32:0/1405457715 wait complete. 2023-11-24T07:58:52.822 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:58:53.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:58:38.060811+0000', 'modified': '2023-11-24T07:58:41.071393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6837/1588575159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6835', 'nonce': 1588575159}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1588575159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:58:53.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:58:53.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-24T07:58:53.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.367+0000 7f6813297700 1 Processor -- start 2023-11-24T07:58:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.367+0000 7f6813297700 1 -- start start 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6813297700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f680c15bef0 0x7f680c15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6813297700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f680c0afe70 0x7f680c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6813297700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c06dd80 con 0x7f680c0b4b40 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6813297700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c15f880 con 0x7f680c15bef0 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6813297700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c15ed90 con 0x7f680c0afe70 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f680c15bef0 0x7f680c15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f680c15bef0 0x7f680c15e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57094/0 (socket says 172.21.15.32:57094) 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 -- 172.21.15.32:0/2511560893 learned_addr learned my addr 172.21.15.32:0/2511560893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 -- 172.21.15.32:0/2511560893 >> v2:172.21.15.123:3300/0 conn(0x7f680c0afe70 msgr2=0x7f680c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6810832700 1 --2- 172.21.15.32:0/2511560893 >> v2:172.21.15.123:3300/0 conn(0x7f680c0afe70 0x7f680c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 --2- 172.21.15.32:0/2511560893 >> v2:172.21.15.123:3300/0 conn(0x7f680c0afe70 0x7f680c0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 -- 172.21.15.32:0/2511560893 >> v1:172.21.15.123:6789/0 conn(0x7f680c0b4b40 legacy=0x7f680c1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 -- 172.21.15.32:0/2511560893 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c15c2d0 con 0x7f680c15bef0 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6810832700 1 --2- 172.21.15.32:0/2511560893 >> v2:172.21.15.123:3300/0 conn(0x7f680c0afe70 0x7f680c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.368+0000 7f6811033700 1 --2- 172.21.15.32:0/2511560893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c15bef0 0x7f680c15e850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f680c071790 tx=0x7f6808009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176ba488da7b5ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6803fff700 1 -- 172.21.15.32:0/2511560893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808004230 con 0x7f680c15bef0 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6803fff700 1 -- 172.21.15.32:0/2511560893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808004390 con 0x7f680c15bef0 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6803fff700 1 -- 172.21.15.32:0/2511560893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080036a0 con 0x7f680c15bef0 2023-11-24T07:58:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 -- 172.21.15.32:0/2511560893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c15bef0 msgr2=0x7f680c15e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 --2- 172.21.15.32:0/2511560893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c15bef0 0x7f680c15e850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f680c071790 tx=0x7f6808009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 -- 172.21.15.32:0/2511560893 shutdown_connections 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 --2- 172.21.15.32:0/2511560893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c15bef0 0x7f680c15e850 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 --2- 172.21.15.32:0/2511560893 >> v2:172.21.15.123:3300/0 conn(0x7f680c0afe70 0x7f680c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 -- 172.21.15.32:0/2511560893 >> 172.21.15.32:0/2511560893 conn(0x7f680c071ac0 msgr2=0x7f680c071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 -- 172.21.15.32:0/2511560893 shutdown_connections 2023-11-24T07:58:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.369+0000 7f6813297700 1 -- 172.21.15.32:0/2511560893 wait complete. 2023-11-24T07:58:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.371+0000 7f6813297700 1 Processor -- start 2023-11-24T07:58:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.371+0000 7f6813297700 1 -- start start 2023-11-24T07:58:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6813297700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c0afe70 0x7f680c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6813297700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 0x7f680c14a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6813297700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f680c15bef0 0x7f680c1f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6813297700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1e72f0 con 0x7f680c0b4b40 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6813297700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0cf6c0 con 0x7f680c0afe70 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6813297700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0cf9a0 con 0x7f680c15bef0 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 0x7f680c14a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 0x7f680c14a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57102/0 (socket says 172.21.15.32:57102) 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 -- 172.21.15.32:0/3126847497 learned_addr learned my addr 172.21.15.32:0/3126847497 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6811033700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c0afe70 0x7f680c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6811834700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f680c15bef0 0x7f680c1f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 -- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f680c15bef0 msgr2=0x7f680c1f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f680c15bef0 0x7f680c1f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 -- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c0afe70 msgr2=0x7f680c0cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c0afe70 0x7f680c0cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 -- 172.21.15.32:0/3126847497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808009550 con 0x7f680c0b4b40 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6811834700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f680c15bef0 0x7f680c1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6810832700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 0x7f680c14a320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c7f0 tx=0x7f67fc00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6801ffb700 1 -- 172.21.15.32:0/3126847497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc011800 con 0x7f680c0b4b40 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.372+0000 7f6801ffb700 1 -- 172.21.15.32:0/3126847497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc011960 con 0x7f680c0b4b40 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.373+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c0cfc80 con 0x7f680c0b4b40 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.373+0000 7f6801ffb700 1 -- 172.21.15.32:0/3126847497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc011cf0 con 0x7f680c0b4b40 2023-11-24T07:58:53.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.373+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c1f82a0 con 0x7f680c0b4b40 2023-11-24T07:58:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.374+0000 7f6801ffb700 1 -- 172.21.15.32:0/3126847497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67fc018360 con 0x7f680c0b4b40 2023-11-24T07:58:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.375+0000 7f6801ffb700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67f006aed0 0x7f67f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.375+0000 7f6801ffb700 1 -- 172.21.15.32:0/3126847497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67fc089570 con 0x7f680c0b4b40 2023-11-24T07:58:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.375+0000 7f6811033700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67f006aed0 0x7f67f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.376+0000 7f6811033700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67f006aed0 0x7f67f006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6808009130 tx=0x7f6808008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.377+0000 7f681f490b80 1 Processor -- start 2023-11-24T07:58:53.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.377+0000 7f681f490b80 1 -- start start 2023-11-24T07:58:53.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f681f490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fd2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48fa1ed0 0x55bc48fd2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f89490 0x55bc48fb5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f681f490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48eff770 con 0x55bc48f8c080 2023-11-24T07:58:53.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48ee6750 con 0x55bc48f89490 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc48ee4070 con 0x55bc48fa1ed0 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fd2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fd2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57108/0 (socket says 172.21.15.32:57108) 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 -- 172.21.15.32:0/1001386768 learned_addr learned my addr 172.21.15.32:0/1001386768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811834700 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f89490 0x55bc48fb5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6810832700 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48fa1ed0 0x55bc48fd2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 -- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48fa1ed0 msgr2=0x55bc48fd2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48fa1ed0 0x55bc48fd2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 -- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f89490 msgr2=0x55bc48fb5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f89490 0x55bc48fb5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 -- 172.21.15.32:0/1001386768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc48d5c5e0 con 0x55bc48f8c080 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f6811033700 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fd2840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6808009fd0 tx=0x7f6808010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40713b9b923d1626 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f67f6ffd700 1 -- 172.21.15.32:0/1001386768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808020070 con 0x55bc48f8c080 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f67f6ffd700 1 -- 172.21.15.32:0/1001386768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808003da0 con 0x55bc48f8c080 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.378+0000 7f67f6ffd700 1 -- 172.21.15.32:0/1001386768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808012880 con 0x55bc48f8c080 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 -- 172.21.15.32:0/1001386768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 msgr2=0x55bc48fd2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fd2840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6808009fd0 tx=0x7f6808010000 comp rx=0 tx=0).stop 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 -- 172.21.15.32:0/1001386768 shutdown_connections 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f89490 0x55bc48fb5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fd2840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 --2- 172.21.15.32:0/1001386768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48fa1ed0 0x55bc48fd2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 -- 172.21.15.32:0/1001386768 >> 172.21.15.32:0/1001386768 conn(0x55bc48eba310 msgr2=0x55bc48fd0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 -- 172.21.15.32:0/1001386768 shutdown_connections 2023-11-24T07:58:53.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.379+0000 7f681f490b80 1 -- 172.21.15.32:0/1001386768 wait complete. 2023-11-24T07:58:53.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.380+0000 7f681f490b80 1 -- start start 2023-11-24T07:58:53.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.380+0000 7f681f490b80 1 Processor -- start 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 0x55bc48fcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f8c080 0x55bc48f80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48f80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48edf960 con 0x55bc48f89490 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48e99510 con 0x55bc48fa1ed0 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc48e991b0 con 0x55bc48f8c080 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 10 client.0 _ll_register_callbacks cb 0x55bc48dc4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 0x55bc48fcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 0x55bc48fcf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57114/0 (socket says 172.21.15.32:57114) 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 -- 172.21.15.32:0/2439069079 learned_addr learned my addr 172.21.15.32:0/2439069079 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:53.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811834700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48f80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6810832700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f8c080 0x55bc48f80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 -- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f8c080 msgr2=0x55bc48f80330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f8c080 0x55bc48f80330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 -- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 msgr2=0x55bc48f80870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48f80870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 -- 172.21.15.32:0/2439069079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c15c2d0 con 0x55bc48f89490 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f6811033700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 0x55bc48fcf970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6808003b30 tx=0x7f68080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3307ea894d4edfe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f67ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f67ebfff700 1 -- 172.21.15.32:0/2439069079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808012c90 con 0x55bc48f89490 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f67ebfff700 1 -- 172.21.15.32:0/2439069079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6808012df0 con 0x55bc48f89490 2023-11-24T07:58:53.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc48f7ea40 con 0x55bc48f89490 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f67ebfff700 1 -- 172.21.15.32:0/2439069079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6808005a20 con 0x55bc48f89490 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 10 client.4677 fetch_fsmap learned FSMap version 8 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.381+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55bc48d08d90 con 0x55bc48f89490 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f67ebfff700 1 -- 172.21.15.32:0/2439069079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808042070 con 0x55bc48f89490 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f67ebfff700 1 -- 172.21.15.32:0/2439069079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6808005b80 con 0x55bc48f89490 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f681f490b80 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f681f490b80 10 client.4677 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f681f490b80 20 client.4677 populate_metadata read hostname 'smithi032' 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f681f490b80 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fe4040 0x55bc48fa0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f681f490b80 4 client.4677 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 --> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc48f5a840 con 0x55bc48fe4040 2023-11-24T07:58:53.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f6810832700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fe4040 0x55bc48fa0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f67ebfff700 10 client.4677 ms_handle_connect on v2:172.21.15.123:6835/1588575159 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.382+0000 7f6810832700 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fe4040 0x55bc48fa0d30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c815bf62f8fa8e server_cookie=2efbceeeefaed194 in_seq=0 out_seq=0 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f67ebfff700 1 -- 172.21.15.32:0/2439069079 <== mds.0 v2:172.21.15.123:6835/1588575159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc48f5a840 con 0x55bc48fe4040 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f67ebfff700 10 client.4677 handle_command_reply: tid=0 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 client.4677 shutdown 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 msgr2=0x55bc48fcf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 0x55bc48fcf970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6808003b30 tx=0x7f68080047c0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 shutdown_connections 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48f80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fe4040 0x55bc48fa0d30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f89490 0x55bc48fcf970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f67ebfff700 0 client.4677 ms_handle_reset on v2:172.21.15.123:6835/1588575159 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 --2- 172.21.15.32:0/2439069079 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f8c080 0x55bc48f80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 >> 172.21.15.32:0/2439069079 conn(0x55bc48eba310 msgr2=0x55bc48fd4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 shutdown_connections 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.383+0000 7f681f490b80 1 -- 172.21.15.32:0/2439069079 wait complete. 2023-11-24T07:58:53.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.384+0000 7f681f490b80 20 client.4677 trim_cache size 0 max 16384 2023-11-24T07:58:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.401+0000 7f681f490b80 1 Processor -- start 2023-11-24T07:58:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 -- start start 2023-11-24T07:58:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc48fc69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fc6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48fcc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48d5e8d0 con 0x55bc48f8c080 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48fb6de0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc48fd16b0 con 0x55bc48f89490 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f6810832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fc6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f6810832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fc6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57124/0 (socket says 172.21.15.32:57124) 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.402+0000 7f6810832700 1 -- 172.21.15.32:0/2585683695 learned_addr learned my addr 172.21.15.32:0/2585683695 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6810832700 1 -- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 msgr2=0x55bc48fc69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6810832700 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc48fc69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6810832700 1 -- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 msgr2=0x55bc48fcc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6811834700 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48fcc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6810832700 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48fcc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6810832700 1 -- 172.21.15.32:0/2585683695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc48cf1c10 con 0x55bc48f8c080 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f6810832700 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fc6f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881d8711f4a9985e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f67f77fe700 1 -- 172.21.15.32:0/2585683695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc060590 con 0x55bc48f8c080 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f67f77fe700 1 -- 172.21.15.32:0/2585683695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc087b50 con 0x55bc48f8c080 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f67f77fe700 1 -- 172.21.15.32:0/2585683695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc056630 con 0x55bc48f8c080 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 -- 172.21.15.32:0/2585683695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 msgr2=0x55bc48fc6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fc6f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc013040 comp rx=0 tx=0).stop 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 -- 172.21.15.32:0/2585683695 shutdown_connections 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48fa1ed0 0x55bc48fcc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc48fc69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 --2- 172.21.15.32:0/2585683695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48f8c080 0x55bc48fc6f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 -- 172.21.15.32:0/2585683695 >> 172.21.15.32:0/2585683695 conn(0x55bc48eba310 msgr2=0x55bc48fdd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 -- 172.21.15.32:0/2585683695 shutdown_connections 2023-11-24T07:58:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.403+0000 7f681f490b80 1 -- 172.21.15.32:0/2585683695 wait complete. 2023-11-24T07:58:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.404+0000 7f681f490b80 1 -- start start 2023-11-24T07:58:53.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.404+0000 7f681f490b80 1 Processor -- start 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc49023440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f8c080 0x55bc48fd43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 0x55bc48fb8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48fb9ca0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc48fd4bf0 con 0x55bc48f8c080 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc49022fb0 con 0x55bc48f89490 2023-11-24T07:58:53.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f681f490b80 10 client.0 _ll_register_callbacks cb 0x55bc48dc4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 0x55bc48fb8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 0x55bc48fb8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57132/0 (socket says 172.21.15.32:57132) 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 -- 172.21.15.32:0/3076265478 learned_addr learned my addr 172.21.15.32:0/3076265478 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811033700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc49023440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6810832700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f8c080 0x55bc48fd43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 -- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 msgr2=0x55bc49023440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc49023440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 -- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f8c080 msgr2=0x55bc48fd43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f8c080 0x55bc48fd43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.405+0000 7f6811834700 1 -- 172.21.15.32:0/3076265478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c1f82a0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f6811834700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 0x55bc48fb8f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55bc48fddd50 tx=0x7f6804014bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ebae961e77de6d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f67ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f67ebfff700 1 -- 172.21.15.32:0/3076265478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804005260 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f67ebfff700 1 -- 172.21.15.32:0/3076265478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68040053c0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6808003da0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f67ebfff700 1 -- 172.21.15.32:0/3076265478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68040045a0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 10 client.4683 fetch_fsmap learned FSMap version 8 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55bc48d2f200 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f67ebfff700 1 -- 172.21.15.32:0/3076265478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808003da0 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f67ebfff700 1 -- 172.21.15.32:0/3076265478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6804004700 con 0x55bc48fa1ed0 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 10 client.4683 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T07:58:53.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 20 client.4683 populate_metadata read hostname 'smithi032' 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fc6f10 0x55bc48fd1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 4 client.4683 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 --> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc48ea5f80 con 0x55bc48fc6f10 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.406+0000 7f6811033700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fc6f10 0x55bc48fd1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f67ebfff700 10 client.4683 ms_handle_connect on v2:172.21.15.123:6835/1588575159 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f6811033700 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fc6f10 0x55bc48fd1ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c54a415e0710bd9 server_cookie=138bf017424fc2ff in_seq=0 out_seq=0 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f67ebfff700 1 -- 172.21.15.32:0/3076265478 <== mds.0 v2:172.21.15.123:6835/1588575159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55bc48ea5f80 con 0x55bc48fc6f10 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f67ebfff700 10 client.4683 handle_command_reply: tid=0 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 client.4683 shutdown 2023-11-24T07:58:53.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 msgr2=0x55bc48fb8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 0x55bc48fb8f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55bc48fddd50 tx=0x7f6804014bb0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 shutdown_connections 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:6835/1588575159,v1:172.21.15.123:6837/1588575159] conn(0x55bc48fc6f10 0x55bc48fd1ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bc48fa1ed0 0x55bc48fb8f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55bc48f89490 0x55bc49023440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f67ebfff700 0 client.4683 ms_handle_reset on v2:172.21.15.123:6835/1588575159 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 --2- 172.21.15.32:0/3076265478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc48f8c080 0x55bc48fd43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 >> 172.21.15.32:0/3076265478 conn(0x55bc48eba310 msgr2=0x55bc48fd0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 shutdown_connections 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 1 -- 172.21.15.32:0/3076265478 wait complete. 2023-11-24T07:58:53.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.407+0000 7f681f490b80 20 client.4683 trim_cache size 0 max 16384 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "0b3006fa-b594-4439-bcd2-bb19d4a76534", 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4272, 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 14.758163824, 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 8, 2023-11-24T07:58:53.417 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 21, 2023-11-24T07:58:53.418 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 21, 2023-11-24T07:58:53.418 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 20.255039762999999 2023-11-24T07:58:53.418 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T07:58:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67f006aed0 msgr2=0x7f67f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67f006aed0 0x7f67f006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6808009130 tx=0x7f6808008030 comp rx=0 tx=0).stop 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 msgr2=0x7f680c14a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 0x7f680c14a320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c7f0 tx=0x7f67fc00cb00 comp rx=0 tx=0).stop 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 shutdown_connections 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67f006aed0 0x7f67f006d390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f680c15bef0 0x7f680c1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f680c0afe70 0x7f680c0cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 --2- 172.21.15.32:0/3126847497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680c0b4b40 0x7f680c14a320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 >> 172.21.15.32:0/3126847497 conn(0x7f680c071ac0 msgr2=0x7f680c0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 shutdown_connections 2023-11-24T07:58:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:53.408+0000 7f6813297700 1 -- 172.21.15.32:0/3126847497 wait complete. 2023-11-24T07:58:53.422 INFO:teuthology.run_tasks:Running task kclient... 2023-11-24T07:58:53.444 INFO:tasks.kclient:Mounting kernel clients... 2023-11-24T07:58:53.445 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T07:58:53.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.445 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T07:58:53.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-11-24T07:58:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-11-24T07:58:53.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:58:53.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.533 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T07:58:53.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-11-24T07:58:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-11-24T07:58:53.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:58:53.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.662 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-11-24T07:58:53.755 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-11-24T07:58:53.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:58:53.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.757 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T07:58:53.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-11-24T07:58:53.888 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-11-24T07:58:53.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:58:53.889 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-24T07:58:53.889 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:58:53.889 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:53.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.014+0000 7f7cba750700 1 Processor -- start 2023-11-24T07:58:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.014+0000 7f7cba750700 1 -- start start 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cba750700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb40b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cba750700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cb40cf360 0x7f7cb40b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cba750700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb415a790 con 0x7f7cb40b3da0 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cba750700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb415eec0 con 0x7f7cb40cf360 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cba750700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40cf8b0 con 0x7f7cb415a3b0 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb8ced700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7cb40b3da0 0x7f7cb406dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58852/0 (socket says 172.21.15.32:58852) 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb8ced700 1 -- 172.21.15.32:0/3293498337 learned_addr learned my addr 172.21.15.32:0/3293498337 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb40b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb37fe700 1 --2- 172.21.15.32:0/3293498337 >> v2:172.21.15.123:3300/0 conn(0x7f7cb40cf360 0x7f7cb40b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb2ffd700 1 -- 172.21.15.32:0/3293498337 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168847204 0 0) 0x7f7cb415a790 con 0x7f7cb40b3da0 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb2ffd700 1 -- 172.21.15.32:0/3293498337 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca00035b0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb2ffd700 1 -- 172.21.15.32:0/3293498337 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257344739 0 0) 0x7f7ca00035b0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb37fe700 1 -- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 msgr2=0x7f7cb40b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb37fe700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb40b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.015+0000 7f7cb37fe700 1 -- 172.21.15.32:0/3293498337 >> v1:172.21.15.32:6789/0 conn(0x7f7cb40b3da0 legacy=0x7f7cb406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cb37fe700 1 -- 172.21.15.32:0/3293498337 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40b4c40 con 0x7f7cb40cf360 2023-11-24T07:58:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb40b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cb37fe700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb40b3860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009870 tx=0x7f7ca4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1990eb7a8c8eae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cb2ffd700 1 -- 172.21.15.32:0/3293498337 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0019f0 con 0x7f7cb40cf360 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cb2ffd700 1 -- 172.21.15.32:0/3293498337 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4004030 con 0x7f7cb40cf360 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cb2ffd700 1 -- 172.21.15.32:0/3293498337 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40043c0 con 0x7f7cb40cf360 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cba750700 1 -- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 msgr2=0x7f7cb40b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cba750700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb40b3860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009870 tx=0x7f7ca4009b80 comp rx=0 tx=0).stop 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cba750700 1 -- 172.21.15.32:0/3293498337 shutdown_connections 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cba750700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb40b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cba750700 1 --2- 172.21.15.32:0/3293498337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb40b3860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.016+0000 7f7cba750700 1 -- 172.21.15.32:0/3293498337 >> 172.21.15.32:0/3293498337 conn(0x7f7cb40b1740 msgr2=0x7f7cb40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.017+0000 7f7cba750700 1 -- 172.21.15.32:0/3293498337 shutdown_connections 2023-11-24T07:58:54.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.017+0000 7f7cba750700 1 -- 172.21.15.32:0/3293498337 wait complete. 2023-11-24T07:58:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.018+0000 7f7cba750700 1 Processor -- start 2023-11-24T07:58:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.018+0000 7f7cba750700 1 -- start start 2023-11-24T07:58:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 0x7f7cb41440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb4149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb41499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb414ca30 con 0x7f7cb40b3da0 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb414cc80 con 0x7f7cb40cf360 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb414cf60 con 0x7f7cb415a3b0 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 0x7f7cb41440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 0x7f7cb41440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57142/0 (socket says 172.21.15.32:57142) 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 -- 172.21.15.32:0/1693191794 learned_addr learned my addr 172.21.15.32:0/1693191794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb37fe700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb4149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb8ced700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb41499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 -- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 msgr2=0x7f7cb41499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb41499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 -- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 msgr2=0x7f7cb4149480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb4149480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 -- 172.21.15.32:0/1693191794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0094a0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb3fff700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 0x7f7cb41440b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00ebe0 tx=0x7f7c9c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00c850 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c00c9b0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00cd40 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.019+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca40094a0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.020+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb414d520 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.020+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb40cc030 con 0x7f7cb40b3da0 2023-11-24T07:58:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.021+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c9c013470 con 0x7f7cb40b3da0 2023-11-24T07:58:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.021+0000 7f7cb0ff9700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.021+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c9c089640 con 0x7f7cb40b3da0 2023-11-24T07:58:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.021+0000 7f7cb37fe700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.022+0000 7f7cb37fe700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9406aed0 0x7f7c9406d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009870 tx=0x7f7ca401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.026+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c9c054780 con 0x7f7cb40b3da0 2023-11-24T07:58:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.274+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cb40cbde0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.274+0000 7f7cb0ff9700 1 -- 172.21.15.32:0/1693191794 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7c9c00cea0 con 0x7f7cb40b3da0 2023-11-24T07:58:54.276 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9406aed0 msgr2=0x7f7c9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9406aed0 0x7f7c9406d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009870 tx=0x7f7ca401c040 comp rx=0 tx=0).stop 2023-11-24T07:58:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 msgr2=0x7f7cb41440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 0x7f7cb41440b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00ebe0 tx=0x7f7c9c00eef0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 shutdown_connections 2023-11-24T07:58:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cb415a3b0 0x7f7cb41499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cb40b3da0 0x7f7cb41440b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 --2- 172.21.15.32:0/1693191794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb40cf360 0x7f7cb4149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.276+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 >> 172.21.15.32:0/1693191794 conn(0x7f7cb40b1740 msgr2=0x7f7cb415c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.277+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 shutdown_connections 2023-11-24T07:58:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:54.277+0000 7f7cba750700 1 -- 172.21.15.32:0/1693191794 wait complete. 2023-11-24T07:58:54.290 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:54.291 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:54.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.291 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-11-24T07:58:54.309 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:54.309 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:54.309 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:54.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:54.309 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:54.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42381sec preferred_lft 42381sec 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:54.310 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:54.311 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:54.311 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T07:58:54.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.311 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:54.311 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:54.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-11-24T07:58:54.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-11-24T07:58:54.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-11-24T07:58:54.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T07:58:54.312 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:54.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.484 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:54.514 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-11-24T07:58:54.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.515 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-11-24T07:58:54.581 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-11-24T07:58:54.581 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:54.581 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:58:54.581 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:58:54.581 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:54.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.582 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:54.582 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:54.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:58:54.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:58:54.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:58:54.583 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:54.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.821 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T07:58:54.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.838 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-11-24T07:58:54.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.894 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:54.894 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:54.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:58:54.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T07:58:54.895 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:54.991 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T07:58:54.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:54.991 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:54.991 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:54.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T07:58:54.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T07:58:54.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T07:58:54.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T07:58:54.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T07:58:54.992 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:55.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.198 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:55.198 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:55.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-11-24T07:58:55.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T07:58:55.199 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:55.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:55.303 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:55.325 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:55.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:55.325 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:55.382 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:58:55.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:55.475 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:55.476 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:55.649 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:55.649 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:55.686 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:55.686 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-11-24T07:58:55.687 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0b3006fa-b594-4439-bcd2-bb19d4a76534" will pass to kernel 2023-11-24T07:58:55.693 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:55.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:55.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.729 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:55.817 INFO:teuthology.orchestra.run.smithi032.stdout:client4692 2023-11-24T07:58:55.818 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:55.818 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:55.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.835 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:55.924 INFO:teuthology.orchestra.run.smithi032.stdout:client4692 2023-11-24T07:58:55.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.925 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:55.979 INFO:teuthology.orchestra.run.smithi032.stdout:client4692 2023-11-24T07:58:55.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.979 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0b3006fa-b594-4439-bcd2-bb19d4a76534.client4692/status') 2023-11-24T07:58:56.035 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4692 (3)192.168.0.1:0/20715887 2023-11-24T07:58:56.035 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-11-24T07:58:56.036 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-11-24T07:58:56.036 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-11-24T07:58:56.036 INFO:teuthology.orchestra.run.smithi032.stderr:67 bytes copied, 3.3487e-05 s, 2.0 MB/s 2023-11-24T07:58:56.037 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-24T07:58:56.037 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:58:56.037 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T07:58:56.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.160+0000 7f7617f16700 1 Processor -- start 2023-11-24T07:58:56.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7617f16700 1 -- start start 2023-11-24T07:58:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7617f16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 0x7f76100cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7617f16700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76100cd5d0 con 0x7f761015c030 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7617f16700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76100cd870 con 0x7f76100ae910 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7617f16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76100cdb50 con 0x7f76100d0780 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7615cb2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f761015c030 0x7f76100d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58866/0 (socket says 172.21.15.32:58866) 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.161+0000 7f7615cb2700 1 -- 172.21.15.32:0/2375024224 learned_addr learned my addr 172.21.15.32:0/2375024224 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f76154b1700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 0x7f76100cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466508905 0 0) 0x7f76100cd5d0 con 0x7f761015c030 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75fc0035b0 con 0x7f761015c030 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169700707 0 0) 0x7f76100cd870 con 0x7f76100ae910 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76100cd5d0 con 0x7f76100ae910 2023-11-24T07:58:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3863726399 0 0) 0x7f75fc0035b0 con 0x7f761015c030 2023-11-24T07:58:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 msgr2=0x7f76100cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 0x7f76100cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 >> v1:172.21.15.123:6789/0 conn(0x7f76100ae910 legacy=0x7f76100cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100cde30 con 0x7f761015c030 2023-11-24T07:58:56.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f760c001670 con 0x7f761015c030 2023-11-24T07:58:56.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 >> v1:172.21.15.32:6789/0 conn(0x7f761015c030 legacy=0x7f76100d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75fc003bc0 0x7f75fc003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc004850 0x7f75fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 0x7f75fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76100cdb50 con 0x7f75fc003bc0 2023-11-24T07:58:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc00bac0 con 0x7f75fc004850 2023-11-24T07:58:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc00bc80 con 0x7f76100d0780 2023-11-24T07:58:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f7615cb2700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75fc003bc0 0x7f75fc003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f76154b1700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc004850 0x7f75fc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.162+0000 7f76164b3700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 0x7f75fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7615cb2700 1 -- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 msgr2=0x7f75fc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7615cb2700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f76100d0780 0x7f75fc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7615cb2700 1 -- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc004850 msgr2=0x7f75fc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7615cb2700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc004850 0x7f75fc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7615cb2700 1 -- 172.21.15.32:0/2375024224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc0035b0 con 0x7f75fc003bc0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7615cb2700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75fc003bc0 0x7f75fc003fc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c110 tx=0x7f75fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7a25cae6a82a58 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c00ccb0 con 0x7f75fc003bc0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c00ce10 con 0x7f75fc003bc0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7614cb0700 1 -- 172.21.15.32:0/2375024224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c005ba0 con 0x7f75fc003bc0 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7617f16700 1 -- 172.21.15.32:0/2375024224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75fc003bc0 msgr2=0x7f75fc003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7617f16700 1 --2- 172.21.15.32:0/2375024224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75fc003bc0 0x7f75fc003fc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c110 tx=0x7f75fc005880 comp rx=0 tx=0).stop 2023-11-24T07:58:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f76164b3700 1 -- 172.21.15.32:0/2375024224 reap_dead start 2023-11-24T07:58:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7617f16700 1 -- 172.21.15.32:0/2375024224 shutdown_connections 2023-11-24T07:58:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.163+0000 7f7617f16700 1 -- 172.21.15.32:0/2375024224 >> 172.21.15.32:0/2375024224 conn(0x7f76100730a0 msgr2=0x7f761009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.164+0000 7f7617f16700 1 -- 172.21.15.32:0/2375024224 shutdown_connections 2023-11-24T07:58:56.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.164+0000 7f7617f16700 1 -- 172.21.15.32:0/2375024224 wait complete. 2023-11-24T07:58:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.165+0000 7f7617f16700 1 Processor -- start 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.165+0000 7f7617f16700 1 -- start start 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc003bc0 0x7f76100c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f761006c870 0x7f761006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 0x7f76100cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761015ea70 con 0x7f76100cb0b0 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761015de80 con 0x7f75fc003bc0 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761015e0d0 con 0x7f761006c870 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 0x7f76100cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 0x7f76100cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57170/0 (socket says 172.21.15.32:57170) 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 -- 172.21.15.32:0/3618231258 learned_addr learned my addr 172.21.15.32:0/3618231258 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7615cb2700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc003bc0 0x7f76100c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76154b1700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f761006c870 0x7f761006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 -- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f761006c870 msgr2=0x7f761006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f761006c870 0x7f761006bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 -- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc003bc0 msgr2=0x7f76100c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc003bc0 0x7f76100c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 -- 172.21.15.32:0/3618231258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761015e3b0 con 0x7f76100cb0b0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76164b3700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 0x7f76100cb530 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f761009b7c0 tx=0x7f760800d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760800dbf0 con 0x7f76100cb0b0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760800dd70 con 0x7f76100cb0b0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761015e690 con 0x7f76100cb0b0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760800dee0 con 0x7f76100cb0b0 2023-11-24T07:58:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.166+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76100cba70 con 0x7f76100cb0b0 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.167+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76100751e0 con 0x7f76100cb0b0 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.168+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7608003c40 con 0x7f76100cb0b0 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.168+0000 7f76027fc700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f75f406aed0 0x7f75f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.168+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76080897e0 con 0x7f76100cb0b0 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.168+0000 7f7615cb2700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f75f406aed0 0x7f75f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.169+0000 7f7615cb2700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f75f406aed0 0x7f75f406d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c110 tx=0x7f760c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.173+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7608054920 con 0x7f76100cb0b0 2023-11-24T07:58:56.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.420+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f761015ebb0 con 0x7f76100cb0b0 2023-11-24T07:58:56.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.420+0000 7f76027fc700 1 -- 172.21.15.32:0/3618231258 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f761015ebb0 con 0x7f76100cb0b0 2023-11-24T07:58:56.422 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:56.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f75f406aed0 msgr2=0x7f75f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f75f406aed0 0x7f75f406d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f760c00c110 tx=0x7f760c007000 comp rx=0 tx=0).stop 2023-11-24T07:58:56.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 msgr2=0x7f76100cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 0x7f76100cb530 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f761009b7c0 tx=0x7f760800d660 comp rx=0 tx=0).stop 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 shutdown_connections 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f75f406aed0 0x7f75f406d390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cb0b0 0x7f76100cb530 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75fc003bc0 0x7f76100c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 --2- 172.21.15.32:0/3618231258 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f761006c870 0x7f761006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.422+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 >> 172.21.15.32:0/3618231258 conn(0x7f76100730a0 msgr2=0x7f76100b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.423+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 shutdown_connections 2023-11-24T07:58:56.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:56.423+0000 7f7617f16700 1 -- 172.21.15.32:0/3618231258 wait complete. 2023-11-24T07:58:56.436 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:58:56.436 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:56.436 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-11-24T07:58:56.436 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T07:58:56.437 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T07:58:56.437 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:58:56.437 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:58:56.437 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:56.437 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T07:58:56.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.437 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-11-24T07:58:56.455 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42379sec preferred_lft 42379sec 2023-11-24T07:58:56.456 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 82:17:03:92:a2:96 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::b2:e9ff:fe75:3b63/64 scope link 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 82:17:03:92:a2:96 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::8017:3ff:fe92:a296/64 scope link tentative 2023-11-24T07:58:56.457 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.458 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:56.521 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-11-24T07:58:56.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.522 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-11-24T07:58:56.581 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.582 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:56.583 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:56.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:58:56.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:58:56.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:58:56.583 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:56.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.688 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T07:58:56.705 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:58:56.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.706 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-11-24T07:58:56.763 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T07:58:56.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.763 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:56.763 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:56.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:58:56.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T07:58:56.764 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:56.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T07:58:56.895 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:56.895 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:56.895 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:56.895 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether da:8a:d2:04:48:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::d88a:d2ff:fe04:486a/64 scope link 2023-11-24T07:58:56.896 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:56.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T07:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether da:8a:d2:04:48:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::d88a:d2ff:fe04:486a/64 scope link 2023-11-24T07:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:57.005 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T07:58:57.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.005 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T07:58:57.006 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:57.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.252 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:58:57.252 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:58:57.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2023-11-24T07:58:57.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T07:58:57.252 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:58:57.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:57.370 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T07:58:57.391 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T07:58:57.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:57.391 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T07:58:57.448 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-24T07:58:57.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:57.535 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:57.536 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:57.566 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:57.566 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:58:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 2@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:58:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:58:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:57.567 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 2@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:57.640 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:57.641 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:57.641 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-11-24T07:58:57.641 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-11-24T07:58:57.641 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=0b3006fa-b594-4439-bcd2-bb19d4a76534" will pass to kernel 2023-11-24T07:58:57.642 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:57.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T07:58:57.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.670 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:57.744 INFO:teuthology.orchestra.run.smithi032.stdout:client4594 2023-11-24T07:58:57.744 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:58:57.744 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:57.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.802 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:57.875 INFO:teuthology.orchestra.run.smithi032.stdout:client4594 2023-11-24T07:58:57.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.875 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:57.949 INFO:teuthology.orchestra.run.smithi032.stdout:client4594 2023-11-24T07:58:57.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.949 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0b3006fa-b594-4439-bcd2-bb19d4a76534.client4594/status') 2023-11-24T07:58:58.030 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4594 (3)172.21.15.32:0/1893608241 2023-11-24T07:58:58.030 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-11-24T07:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-11-24T07:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-11-24T07:58:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 2.4617e-05 s, 2.8 MB/s 2023-11-24T07:58:58.032 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-24T07:58:58.032 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:58:58.032 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:58.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.195+0000 7efecca23700 1 Processor -- start 2023-11-24T07:58:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.195+0000 7efecca23700 1 -- start start 2023-11-24T07:58:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efecca23700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec80622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efecca23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efec80af1e0 0x7efec80b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efecca23700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec815c540 con 0x7efec815c160 2023-11-24T07:58:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efecca23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec80b2ee0 con 0x7efec80af1e0 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efecca23700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec80add40 con 0x7efec80b04b0 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efec659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efec815c160 0x7efec80afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58890/0 (socket says 172.21.15.32:58890) 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efec6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efec80af1e0 0x7efec80b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.196+0000 7efec659c700 1 -- 172.21.15.32:0/3982304502 learned_addr learned my addr 172.21.15.32:0/3982304502 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec5d9b700 1 --2- 172.21.15.32:0/3982304502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec80622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec6d9d700 1 -- 172.21.15.32:0/3982304502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 msgr2=0x7efec80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec6d9d700 1 --2- 172.21.15.32:0/3982304502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec80622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec6d9d700 1 -- 172.21.15.32:0/3982304502 >> v1:172.21.15.32:6789/0 conn(0x7efec815c160 legacy=0x7efec80afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec6d9d700 1 -- 172.21.15.32:0/3982304502 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80ad410 con 0x7efec80af1e0 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec559a700 1 -- 172.21.15.32:0/3982304502 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568525967 0 0) 0x7efec815c540 con 0x7efec815c160 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec6d9d700 1 --2- 172.21.15.32:0/3982304502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80b2870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efec0009d90 tx=0x7efec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb84805261fb1311 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec559a700 1 -- 172.21.15.32:0/3982304502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000e070 con 0x7efec80af1e0 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec559a700 1 -- 172.21.15.32:0/3982304502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000bbd0 con 0x7efec80af1e0 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efec559a700 1 -- 172.21.15.32:0/3982304502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0012430 con 0x7efec80af1e0 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efecca23700 1 -- 172.21.15.32:0/3982304502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 msgr2=0x7efec80b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.197+0000 7efecca23700 1 --2- 172.21.15.32:0/3982304502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80b2870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efec0009d90 tx=0x7efec000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.198+0000 7efecca23700 1 -- 172.21.15.32:0/3982304502 shutdown_connections 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.198+0000 7efecca23700 1 --2- 172.21.15.32:0/3982304502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80b2870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.198+0000 7efecca23700 1 --2- 172.21.15.32:0/3982304502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec80622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.198+0000 7efecca23700 1 -- 172.21.15.32:0/3982304502 >> 172.21.15.32:0/3982304502 conn(0x7efec80b57b0 msgr2=0x7efec80b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.198+0000 7efecca23700 1 -- 172.21.15.32:0/3982304502 shutdown_connections 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.198+0000 7efecca23700 1 -- 172.21.15.32:0/3982304502 wait complete. 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.199+0000 7efecca23700 1 Processor -- start 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.199+0000 7efecca23700 1 -- start start 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec815c160 0x7efec814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec815cb00 con 0x7efec80af1e0 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80cac20 con 0x7efec815c160 2023-11-24T07:58:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec81524e0 con 0x7efec80b04b0 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57184/0 (socket says 172.21.15.32:57184) 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 -- 172.21.15.32:0/2101015357 learned_addr learned my addr 172.21.15.32:0/2101015357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec5d9b700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec6d9d700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec815c160 0x7efec814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 -- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 msgr2=0x7efec8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec8099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 -- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec815c160 msgr2=0x7efec814cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec815c160 0x7efec814cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 -- 172.21.15.32:0/2101015357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009a70 con 0x7efec80af1e0 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec6d9d700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec815c160 0x7efec814cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:58.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efec659c700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80cd5f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efebc0011e0 tx=0x7efebc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc004c80 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc004e00 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec815c540 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.200+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc007310 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.201+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec80cae70 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.201+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec8067f60 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.202+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efebc007470 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.202+0000 7efeb6ffd700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efea806aed0 0x7efea806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.202+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efebc089410 con 0x7efec80af1e0 2023-11-24T07:58:58.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.203+0000 7efec5d9b700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efea806aed0 0x7efea806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.203+0000 7efec5d9b700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efea806aed0 0x7efea806d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efeb800b9b0 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.207+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efebc0544c0 con 0x7efec80af1e0 2023-11-24T07:58:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.453+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efec8154cb0 con 0x7efec80af1e0 2023-11-24T07:58:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.453+0000 7efeb6ffd700 1 -- 172.21.15.32:0/2101015357 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7efebc057ae0 con 0x7efec80af1e0 2023-11-24T07:58:58.455 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efea806aed0 msgr2=0x7efea806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efea806aed0 0x7efea806d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efeb800b9b0 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2023-11-24T07:58:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 msgr2=0x7efec80cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80cd5f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efebc0011e0 tx=0x7efebc00cae0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 shutdown_connections 2023-11-24T07:58:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efea806aed0 0x7efea806d390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec815c160 0x7efec814cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec80af1e0 0x7efec80cd5f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 --2- 172.21.15.32:0/2101015357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efec80b04b0 0x7efec8099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 >> 172.21.15.32:0/2101015357 conn(0x7efec80b57b0 msgr2=0x7efec806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 shutdown_connections 2023-11-24T07:58:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:58:58.455+0000 7efecca23700 1 -- 172.21.15.32:0/2101015357 wait complete. 2023-11-24T07:58:58.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:58:58.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:58.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-11-24T07:58:58.469 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:58:58.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:58.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:58:58.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:58:58.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:58.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:58.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:58.471 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:58.492 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42385sec preferred_lft 42385sec 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5e64:37:eb07:4486/64 scope link noprefixroute 2023-11-24T07:58:58.493 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:58.494 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T07:58:58.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T07:58:58.494 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:58:58.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:58.674 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:58.706 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-11-24T07:58:58.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:58.708 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-11-24T07:58:58.774 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-11-24T07:58:58.774 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:58.775 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:58:58.775 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:58:58.775 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:58.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:58.776 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:58:58.776 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:58:58.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:58:58.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:58:58.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:58:58.776 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:58:59.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.072 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T07:58:59.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.090 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-11-24T07:58:59.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.148 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:58:59.149 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:58:59.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:58:59.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T07:58:59.149 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:58:59.242 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T07:58:59.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T07:58:59.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T07:58:59.244 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:58:59.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.469 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:58:59.469 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:58:59.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-11-24T07:58:59.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T07:58:59.470 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:58:59.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:59.570 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:59.590 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:59.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:59.590 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:59.647 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:58:59.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:59.747 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:59.747 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:59.847 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 1@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:59.848 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:59.880 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:59.881 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:59.881 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:58:59.881 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 1@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:58:59.881 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:58:59.881 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:59.881 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:59.882 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:59.882 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-11-24T07:58:59.882 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-11-24T07:58:59.882 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0b3006fa-b594-4439-bcd2-bb19d4a76534" will pass to kernel 2023-11-24T07:58:59.884 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:59.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:59.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:59.914 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:59.985 INFO:teuthology.orchestra.run.smithi123.stdout:client4597 2023-11-24T07:58:59.986 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:58:59.986 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:00.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.043 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:00.114 INFO:teuthology.orchestra.run.smithi123.stdout:client4597 2023-11-24T07:59:00.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.114 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:00.186 INFO:teuthology.orchestra.run.smithi123.stdout:client4597 2023-11-24T07:59:00.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.186 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0b3006fa-b594-4439-bcd2-bb19d4a76534.client4597/status') 2023-11-24T07:59:00.266 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4597 (3)192.168.0.1:0/3368528398 2023-11-24T07:59:00.266 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2023-11-24T07:59:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2023-11-24T07:59:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2023-11-24T07:59:00.266 INFO:teuthology.orchestra.run.smithi123.stderr:69 bytes copied, 2.2268e-05 s, 3.1 MB/s 2023-11-24T07:59:00.267 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-24T07:59:00.267 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:59:00.268 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T07:59:00.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:59:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.406+0000 7f6e48371700 1 Processor -- start 2023-11-24T07:59:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.406+0000 7f6e48371700 1 -- start start 2023-11-24T07:59:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e48371700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4015e080 0x7f6e400b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e48371700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400ae430 con 0x7f6e4006c710 2023-11-24T07:59:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e48371700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400ae710 con 0x7f6e400af4d0 2023-11-24T07:59:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e48371700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400af0b0 con 0x7f6e4015e080 2023-11-24T07:59:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4690e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6e4006c710 0x7f6e4006e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58908/0 (socket says 172.21.15.32:58908) 2023-11-24T07:59:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4690e700 1 -- 172.21.15.32:0/4183127071 learned_addr learned my addr 172.21.15.32:0/4183127071 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4610d700 1 --2- 172.21.15.32:0/4183127071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4015e080 0x7f6e400b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309134984 0 0) 0x7f6e400ae430 con 0x7f6e4006c710 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c0035b0 con 0x7f6e4006c710 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046825093 0 0) 0x7f6e400ae710 con 0x7f6e400af4d0 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e400ae430 con 0x7f6e400af4d0 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4610d700 1 -- 172.21.15.32:0/4183127071 >> v1:172.21.15.123:6789/0 conn(0x7f6e400af4d0 legacy=0x7f6e4006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4610d700 1 -- 172.21.15.32:0/4183127071 >> v1:172.21.15.32:6789/0 conn(0x7f6e4006c710 legacy=0x7f6e4006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.407+0000 7f6e4610d700 1 -- 172.21.15.32:0/4183127071 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400ae9f0 con 0x7f6e4015e080 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e4610d700 1 --2- 172.21.15.32:0/4183127071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4015e080 0x7f6e400b5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d0bc1fc89aaf9fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011830 con 0x7f6e4015e080 2023-11-24T07:59:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c011990 con 0x7f6e4015e080 2023-11-24T07:59:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e4510b700 1 -- 172.21.15.32:0/4183127071 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011d20 con 0x7f6e4015e080 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 -- 172.21.15.32:0/4183127071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4015e080 msgr2=0x7f6e400b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 --2- 172.21.15.32:0/4183127071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4015e080 0x7f6e400b5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 -- 172.21.15.32:0/4183127071 shutdown_connections 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 --2- 172.21.15.32:0/4183127071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e4015e080 0x7f6e400b5050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 -- 172.21.15.32:0/4183127071 >> 172.21.15.32:0/4183127071 conn(0x7f6e40071b90 msgr2=0x7f6e40071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 -- 172.21.15.32:0/4183127071 shutdown_connections 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.408+0000 7f6e48371700 1 -- 172.21.15.32:0/4183127071 wait complete. 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 Processor -- start 2023-11-24T07:59:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 -- start start 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 0x7f6e4014a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e400af4d0 0x7f6e40155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4015e080 0x7f6e40155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4014ad80 con 0x7f6e4006c710 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40153e60 con 0x7f6e4015e080 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e48371700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40154cc0 con 0x7f6e400af4d0 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e4610d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 0x7f6e4014a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e4610d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 0x7f6e4014a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57198/0 (socket says 172.21.15.32:57198) 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.410+0000 7f6e4610d700 1 -- 172.21.15.32:0/4196316387 learned_addr learned my addr 172.21.15.32:0/4196316387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4590c700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e400af4d0 0x7f6e40155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4690e700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4015e080 0x7f6e40155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4610d700 1 -- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e400af4d0 msgr2=0x7f6e40155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4610d700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e400af4d0 0x7f6e40155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4610d700 1 -- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4015e080 msgr2=0x7f6e40155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4610d700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4015e080 0x7f6e40155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4610d700 1 -- 172.21.15.32:0/4196316387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c0094a0 con 0x7f6e4006c710 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4690e700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4015e080 0x7f6e40155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e4610d700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 0x7f6e4014a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c014040 tx=0x7f6e3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c01a760 con 0x7f6e4006c710 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c01a8c0 con 0x7f6e4006c710 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e401540b0 con 0x7f6e4006c710 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c01ac30 con 0x7f6e4006c710 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.411+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e40154670 con 0x7f6e4006c710 2023-11-24T07:59:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.412+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e400753f0 con 0x7f6e4006c710 2023-11-24T07:59:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.413+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e3c022070 con 0x7f6e4006c710 2023-11-24T07:59:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.413+0000 7f6e36ffd700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6e2406aed0 0x7f6e2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.413+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e3c08a6b0 con 0x7f6e4006c710 2023-11-24T07:59:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.413+0000 7f6e4590c700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6e2406aed0 0x7f6e2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.414+0000 7f6e4590c700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6e2406aed0 0x7f6e2406d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e4014b060 tx=0x7f6e30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.417+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e3c055910 con 0x7f6e4006c710 2023-11-24T07:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.667+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e400b55d0 con 0x7f6e4006c710 2023-11-24T07:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.667+0000 7f6e36ffd700 1 -- 172.21.15.32:0/4196316387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e3c029060 con 0x7f6e4006c710 2023-11-24T07:59:00.669 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:59:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6e2406aed0 msgr2=0x7f6e2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6e2406aed0 0x7f6e2406d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e4014b060 tx=0x7f6e30009040 comp rx=0 tx=0).stop 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 msgr2=0x7f6e4014a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 0x7f6e4014a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c014040 tx=0x7f6e3c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 shutdown_connections 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6e2406aed0 0x7f6e2406d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e4015e080 0x7f6e40155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e4006c710 0x7f6e4014a840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 --2- 172.21.15.32:0/4196316387 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6e400af4d0 0x7f6e40155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 >> 172.21.15.32:0/4196316387 conn(0x7f6e40071b90 msgr2=0x7f6e400b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 shutdown_connections 2023-11-24T07:59:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:00.669+0000 7f6e48371700 1 -- 172.21.15.32:0/4196316387 wait complete. 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:59:00.683 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T07:59:00.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.684 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-11-24T07:59:00.701 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:00.702 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:00.702 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:00.702 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:00.702 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:00.702 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:00.702 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:59:00.703 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42383sec preferred_lft 42383sec 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5e64:37:eb07:4486/64 scope link noprefixroute 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5e:8e:67:5b:95:d2 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::890:f0ff:fe2b:8732/64 scope link 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:00.704 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:00.705 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5e:8e:67:5b:95:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:59:00.705 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5c8e:67ff:fe5b:95d2/64 scope link 2023-11-24T07:59:00.705 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:00.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.705 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:59:00.767 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-11-24T07:59:00.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.768 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-11-24T07:59:00.828 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-11-24T07:59:00.828 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:59:00.828 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:59:00.828 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:59:00.829 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:59:00.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.829 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:59:00.829 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:59:00.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:59:00.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:59:00.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:59:00.830 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:59:00.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.934 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T07:59:00.950 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:59:00.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:00.950 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-11-24T07:59:01.007 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T07:59:01.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.008 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:59:01.008 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:59:01.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:59:01.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T07:59:01.008 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:59:01.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T07:59:01.135 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:01.135 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 76:5e:02:d3:90:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::745e:2ff:fed3:90a7/64 scope link 2023-11-24T07:59:01.136 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T07:59:01.223 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:01.223 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 76:5e:02:d3:90:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::745e:2ff:fed3:90a7/64 scope link 2023-11-24T07:59:01.224 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:01.235 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T07:59:01.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T07:59:01.236 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:59:01.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.480 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:59:01.481 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:59:01.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2023-11-24T07:59:01.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T07:59:01.481 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:59:01.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:01.587 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T07:59:01.607 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T07:59:01.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:01.607 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T07:59:01.664 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-24T07:59:01.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:01.749 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:01.749 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 3@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:01.776 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:01.805 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:01.805 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:01.806 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:59:01.806 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 3@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:59:01.806 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:59:01.806 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:01.806 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:01.807 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:01.807 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-11-24T07:59:01.807 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-11-24T07:59:01.807 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=0b3006fa-b594-4439-bcd2-bb19d4a76534" will pass to kernel 2023-11-24T07:59:01.810 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:59:01.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T07:59:01.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.841 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:01.913 INFO:teuthology.orchestra.run.smithi123.stdout:client4606 2023-11-24T07:59:01.913 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:01.913 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:01.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:01.971 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:02.042 INFO:teuthology.orchestra.run.smithi123.stdout:client4606 2023-11-24T07:59:02.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:02.043 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:02.113 INFO:teuthology.orchestra.run.smithi123.stdout:client4606 2023-11-24T07:59:02.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:02.114 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0b3006fa-b594-4439-bcd2-bb19d4a76534.client4606/status') 2023-11-24T07:59:02.193 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4606 (3)192.168.0.2:0/3070528858 2023-11-24T07:59:02.193 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2023-11-24T07:59:02.194 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2023-11-24T07:59:02.194 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2023-11-24T07:59:02.194 INFO:teuthology.orchestra.run.smithi123.stderr:69 bytes copied, 3.8043e-05 s, 1.8 MB/s 2023-11-24T07:59:02.195 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-24T07:59:02.209 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T07:59:02.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:02.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.377+0000 7f719454d700 1 Processor -- start 2023-11-24T07:59:02.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.377+0000 7f719454d700 1 -- start start 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f719454d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c072df0 0x7f718c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f719454d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f718c0b0230 0x7f718c0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f719454d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718c0ccde0 con 0x7f718c15ddf0 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f719454d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c0cd0c0 con 0x7f718c0b0230 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f719454d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0cd3a0 con 0x7f718c072df0 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f7192aea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f718c0b0230 0x7f718c0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f7192aea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f718c0b0230 0x7f718c0cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57216/0 (socket says 172.21.15.32:57216) 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.378+0000 7f7192aea700 1 -- 172.21.15.32:0/2837875146 learned_addr learned my addr 172.21.15.32:0/2837875146 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f7191ae8700 1 --2- 172.21.15.32:0/2837875146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c072df0 0x7f718c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f7192aea700 1 -- 172.21.15.32:0/2837875146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c072df0 msgr2=0x7f718c0b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f7192aea700 1 --2- 172.21.15.32:0/2837875146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c072df0 0x7f718c0b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f7192aea700 1 -- 172.21.15.32:0/2837875146 >> v1:172.21.15.32:6789/0 conn(0x7f718c15ddf0 legacy=0x7f718c0726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f7192aea700 1 -- 172.21.15.32:0/2837875146 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c0cd680 con 0x7f718c0b0230 2023-11-24T07:59:02.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f7192aea700 1 --2- 172.21.15.32:0/2837875146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c0b0230 0x7f718c0cacb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f718400d150 tx=0x7f718400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1fa8d0f48a6ff9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f71912e7700 1 -- 172.21.15.32:0/2837875146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184010070 con 0x7f718c0b0230 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f71912e7700 1 -- 172.21.15.32:0/2837875146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718400db00 con 0x7f718c0b0230 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.379+0000 7f71912e7700 1 -- 172.21.15.32:0/2837875146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184016400 con 0x7f718c0b0230 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 -- 172.21.15.32:0/2837875146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c0b0230 msgr2=0x7f718c0cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 --2- 172.21.15.32:0/2837875146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c0b0230 0x7f718c0cacb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f718400d150 tx=0x7f718400d460 comp rx=0 tx=0).stop 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 -- 172.21.15.32:0/2837875146 shutdown_connections 2023-11-24T07:59:02.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 --2- 172.21.15.32:0/2837875146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c0b0230 0x7f718c0cacb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 --2- 172.21.15.32:0/2837875146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c072df0 0x7f718c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 -- 172.21.15.32:0/2837875146 >> 172.21.15.32:0/2837875146 conn(0x7f718c0b57b0 msgr2=0x7f718c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:02.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 -- 172.21.15.32:0/2837875146 shutdown_connections 2023-11-24T07:59:02.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.380+0000 7f719454d700 1 -- 172.21.15.32:0/2837875146 wait complete. 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.381+0000 7f719454d700 1 Processor -- start 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.381+0000 7f719454d700 1 -- start start 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f719454d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 0x7f718c09a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f719454d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f718c0b0230 0x7f718c14d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f719454d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c15ddf0 0x7f718c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f719454d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c15b610 con 0x7f718c072df0 2023-11-24T07:59:02.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f719454d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0b3bf0 con 0x7f718c0b0230 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f719454d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c15b090 con 0x7f718c15ddf0 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 0x7f718c09a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 0x7f718c09a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57218/0 (socket says 172.21.15.32:57218) 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 -- 172.21.15.32:0/3669034073 learned_addr learned my addr 172.21.15.32:0/3669034073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f7192aea700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c15ddf0 0x7f718c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f7191ae8700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f718c0b0230 0x7f718c14d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 -- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c15ddf0 msgr2=0x7f718c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c15ddf0 0x7f718c0b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 -- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f718c0b0230 msgr2=0x7f718c14d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f718c0b0230 0x7f718c14d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.382+0000 7f71922e9700 1 -- 172.21.15.32:0/3669034073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718400bd40 con 0x7f718c072df0 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.383+0000 7f71922e9700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 0x7f718c09a380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f718800c460 tx=0x7f718800c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.383+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71880052d0 con 0x7f718c072df0 2023-11-24T07:59:02.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.383+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7188005450 con 0x7f718c072df0 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.383+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188006aa0 con 0x7f718c072df0 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.383+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c15b330 con 0x7f718c072df0 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.383+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c09a960 con 0x7f718c072df0 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.384+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c067f60 con 0x7f718c072df0 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.384+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7188003680 con 0x7f718c072df0 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.385+0000 7f717effd700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f717006aed0 0x7f717006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.385+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7188088610 con 0x7f718c072df0 2023-11-24T07:59:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.385+0000 7f7191ae8700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f717006aed0 0x7f717006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.385+0000 7f7191ae8700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f717006aed0 0x7f717006d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f718c0cb600 tx=0x7f7180009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.389+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7188053750 con 0x7f718c072df0 2023-11-24T07:59:02.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.592+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f718c074b00 con 0x7f718c072df0 2023-11-24T07:59:02.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.593+0000 7f717effd700 1 -- 172.21.15.32:0/3669034073 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f7188056d70 con 0x7f718c072df0 2023-11-24T07:59:02.595 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:02.595 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f717006aed0 msgr2=0x7f717006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f717006aed0 0x7f717006d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f718c0cb600 tx=0x7f7180009210 comp rx=0 tx=0).stop 2023-11-24T07:59:02.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 msgr2=0x7f718c09a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 0x7f718c09a380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f718800c460 tx=0x7f718800c770 comp rx=0 tx=0).stop 2023-11-24T07:59:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 shutdown_connections 2023-11-24T07:59:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f717006aed0 0x7f717006d390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718c15ddf0 0x7f718c0b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f718c072df0 0x7f718c09a380 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 --2- 172.21.15.32:0/3669034073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f718c0b0230 0x7f718c14d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.595+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 >> 172.21.15.32:0/3669034073 conn(0x7f718c0b57b0 msgr2=0x7f718c15ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.596+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 shutdown_connections 2023-11-24T07:59:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.596+0000 7f719454d700 1 -- 172.21.15.32:0/3669034073 wait complete. 2023-11-24T07:59:02.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:02.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 Processor -- start 2023-11-24T07:59:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 -- start start 2023-11-24T07:59:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f05e815a620 0x7f05e80ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e80cc540 0x7f05e806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05e806eb20 0x7f05e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e806efb0 con 0x7f05e806eb20 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80b2f20 con 0x7f05e815a620 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.739+0000 7f05eef98700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80ce690 con 0x7f05e80cc540 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05e806eb20 0x7f05e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e80cc540 0x7f05e806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05e806eb20 0x7f05e806d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57232/0 (socket says 172.21.15.32:57232) 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 -- 172.21.15.32:0/4158356159 learned_addr learned my addr 172.21.15.32:0/4158356159 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ecd34700 1 --2- 172.21.15.32:0/4158356159 >> v2:172.21.15.123:3300/0 conn(0x7f05e815a620 0x7f05e80ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 -- 172.21.15.32:0/4158356159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e80cc540 msgr2=0x7f05e806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 --2- 172.21.15.32:0/4158356159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e80cc540 0x7f05e806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 -- 172.21.15.32:0/4158356159 >> v2:172.21.15.123:3300/0 conn(0x7f05e815a620 msgr2=0x7f05e80ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 --2- 172.21.15.32:0/4158356159 >> v2:172.21.15.123:3300/0 conn(0x7f05e815a620 0x7f05e80ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 -- 172.21.15.32:0/4158356159 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e815aa00 con 0x7f05e806eb20 2023-11-24T07:59:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05e7fff700 1 --2- 172.21.15.32:0/4158356159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e80cc540 0x7f05e806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ecd34700 1 --2- 172.21.15.32:0/4158356159 >> v2:172.21.15.123:3300/0 conn(0x7f05e815a620 0x7f05e80ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05ed535700 1 --2- 172.21.15.32:0/4158356159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e806eb20 0x7f05e806d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05e8072800 tx=0x7f05e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7708a44ed495ae2e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05e77fe700 1 -- 172.21.15.32:0/4158356159 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000e070 con 0x7f05e806eb20 2023-11-24T07:59:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05e77fe700 1 -- 172.21.15.32:0/4158356159 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000b910 con 0x7f05e806eb20 2023-11-24T07:59:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.740+0000 7f05e77fe700 1 -- 172.21.15.32:0/4158356159 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000bcc0 con 0x7f05e806eb20 2023-11-24T07:59:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 -- 172.21.15.32:0/4158356159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e806eb20 msgr2=0x7f05e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 --2- 172.21.15.32:0/4158356159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e806eb20 0x7f05e806d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05e8072800 tx=0x7f05e000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 -- 172.21.15.32:0/4158356159 shutdown_connections 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 --2- 172.21.15.32:0/4158356159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e806eb20 0x7f05e806d3c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 --2- 172.21.15.32:0/4158356159 >> v2:172.21.15.123:3300/0 conn(0x7f05e815a620 0x7f05e80ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 --2- 172.21.15.32:0/4158356159 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e80cc540 0x7f05e806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 -- 172.21.15.32:0/4158356159 >> 172.21.15.32:0/4158356159 conn(0x7f05e80b2050 msgr2=0x7f05e80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 -- 172.21.15.32:0/4158356159 shutdown_connections 2023-11-24T07:59:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.741+0000 7f05eef98700 1 -- 172.21.15.32:0/4158356159 wait complete. 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.742+0000 7f05eef98700 1 Processor -- start 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.742+0000 7f05eef98700 1 -- start start 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05eef98700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e806eb20 0x7f05e81f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05eef98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 0x7f05e80c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05eef98700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e815a620 0x7f05e80ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05eef98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80c6df0 con 0x7f05e80cc540 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05eef98700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80c61e0 con 0x7f05e806eb20 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05eef98700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e80c64c0 con 0x7f05e815a620 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 0x7f05e80c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 0x7f05e80c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57236/0 (socket says 172.21.15.32:57236) 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 -- 172.21.15.32:0/24131878 learned_addr learned my addr 172.21.15.32:0/24131878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05ed535700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e815a620 0x7f05e80ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05ecd34700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e806eb20 0x7f05e81f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 -- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e815a620 msgr2=0x7f05e80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e815a620 0x7f05e80ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 -- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e806eb20 msgr2=0x7f05e81f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e806eb20 0x7f05e81f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05e7fff700 1 -- 172.21.15.32:0/24131878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40094a0 con 0x7f05e80cc540 2023-11-24T07:59:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05ed535700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e815a620 0x7f05e80ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.743+0000 7f05ecd34700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e806eb20 0x7f05e81f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.744+0000 7f05e7fff700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 0x7f05e80c5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05d400ebe0 tx=0x7f05d4009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.744+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4009780 con 0x7f05e80cc540 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.744+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d400c8b0 con 0x7f05e80cc540 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.744+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc0094a0 con 0x7f05e80cc540 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.744+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400cc40 con 0x7f05e80cc540 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.744+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e0009b30 con 0x7f05e80cc540 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.745+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e8068d90 con 0x7f05e80cc540 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.745+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05d4013470 con 0x7f05e80cc540 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.746+0000 7f05e57fa700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f05cc06aed0 0x7f05cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.746+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d40896e0 con 0x7f05e80cc540 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.746+0000 7f05ecd34700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f05cc06aed0 0x7f05cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.746+0000 7f05ecd34700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f05cc06aed0 0x7f05cc06d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009cf0 tx=0x7f05dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.750+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05d4054790 con 0x7f05e80cc540 2023-11-24T07:59:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.955+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05e8068fb0 con 0x7f05e80cc540 2023-11-24T07:59:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.955+0000 7f05e57fa700 1 -- 172.21.15.32:0/24131878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05d4057db0 con 0x7f05e80cc540 2023-11-24T07:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:02.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f05cc06aed0 msgr2=0x7f05cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f05cc06aed0 0x7f05cc06d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009cf0 tx=0x7f05dc008040 comp rx=0 tx=0).stop 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 msgr2=0x7f05e80c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 0x7f05e80c5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05d400ebe0 tx=0x7f05d4009e40 comp rx=0 tx=0).stop 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 shutdown_connections 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f05cc06aed0 0x7f05cc06d390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05e815a620 0x7f05e80ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05e806eb20 0x7f05e81f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 --2- 172.21.15.32:0/24131878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05e80cc540 0x7f05e80c5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.957+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 >> 172.21.15.32:0/24131878 conn(0x7f05e80b2050 msgr2=0x7f05e8156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:02.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.958+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 shutdown_connections 2023-11-24T07:59:02.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:02.958+0000 7f05eef98700 1 -- 172.21.15.32:0/24131878 wait complete. 2023-11-24T07:59:02.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.097+0000 7f29eacfb700 1 Processor -- start 2023-11-24T07:59:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.098+0000 7f29eacfb700 1 -- start start 2023-11-24T07:59:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.098+0000 7f29eacfb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29e415f9e0 0x7f29e40add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.098+0000 7f29eacfb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e406b080 con 0x7f29e4073f90 2023-11-24T07:59:03.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.098+0000 7f29eacfb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e415c290 con 0x7f29e40cf610 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.098+0000 7f29eacfb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e40cf9f0 con 0x7f29e415f9e0 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29e415f9e0 0x7f29e40add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29e415f9e0 0x7f29e40add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57242/0 (socket says 172.21.15.32:57242) 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 -- 172.21.15.32:0/364364605 learned_addr learned my addr 172.21.15.32:0/364364605 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 -- 172.21.15.32:0/364364605 >> v1:172.21.15.123:6789/0 conn(0x7f29e40cf610 legacy=0x7f29e40b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 -- 172.21.15.32:0/364364605 >> v1:172.21.15.32:6789/0 conn(0x7f29e4073f90 legacy=0x7f29e406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 -- 172.21.15.32:0/364364605 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e415ff80 con 0x7f29e415f9e0 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e3fff700 1 --2- 172.21.15.32:0/364364605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e415f9e0 0x7f29e40add40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009920 tx=0x7f29d0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7ff58743a2c382 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e37fe700 1 -- 172.21.15.32:0/364364605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000ba20 con 0x7f29e415f9e0 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e37fe700 1 -- 172.21.15.32:0/364364605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d000bb80 con 0x7f29e415f9e0 2023-11-24T07:59:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.099+0000 7f29e37fe700 1 -- 172.21.15.32:0/364364605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0009e70 con 0x7f29e415f9e0 2023-11-24T07:59:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 -- 172.21.15.32:0/364364605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e415f9e0 msgr2=0x7f29e40add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 --2- 172.21.15.32:0/364364605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e415f9e0 0x7f29e40add40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009920 tx=0x7f29d0009c30 comp rx=0 tx=0).stop 2023-11-24T07:59:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 -- 172.21.15.32:0/364364605 shutdown_connections 2023-11-24T07:59:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 --2- 172.21.15.32:0/364364605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e415f9e0 0x7f29e40add40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 -- 172.21.15.32:0/364364605 >> 172.21.15.32:0/364364605 conn(0x7f29e40d0bf0 msgr2=0x7f29e40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 -- 172.21.15.32:0/364364605 shutdown_connections 2023-11-24T07:59:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.100+0000 7f29eacfb700 1 -- 172.21.15.32:0/364364605 wait complete. 2023-11-24T07:59:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.101+0000 7f29eacfb700 1 Processor -- start 2023-11-24T07:59:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.101+0000 7f29eacfb700 1 -- start start 2023-11-24T07:59:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.102+0000 7f29eacfb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e4073f90 0x7f29e4150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.102+0000 7f29eacfb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 0x7f29e4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.102+0000 7f29eacfb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415f9e0 0x7f29e4152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29eacfb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e415d970 con 0x7f29e40cf610 2023-11-24T07:59:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29eacfb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4157a50 con 0x7f29e415f9e0 2023-11-24T07:59:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29eacfb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4150db0 con 0x7f29e4073f90 2023-11-24T07:59:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 0x7f29e4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 0x7f29e4152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57250/0 (socket says 172.21.15.32:57250) 2023-11-24T07:59:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 -- 172.21.15.32:0/2151533143 learned_addr learned my addr 172.21.15.32:0/2151533143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e9298700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415f9e0 0x7f29e4152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e8a97700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e4073f90 0x7f29e4150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 -- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e4073f90 msgr2=0x7f29e4150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e4073f90 0x7f29e4150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 -- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415f9e0 msgr2=0x7f29e4152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415f9e0 0x7f29e4152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 -- 172.21.15.32:0/2151533143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0009550 con 0x7f29e40cf610 2023-11-24T07:59:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e9298700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415f9e0 0x7f29e4152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e8a97700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e4073f90 0x7f29e4150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e3fff700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 0x7f29e4152a80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29d0014040 tx=0x7f29d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00036a0 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d001e070 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e4154d30 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0003a60 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.103+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e415d2b0 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.104+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e4068fb0 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.105+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29d0003bc0 con 0x7f29e40cf610 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.106+0000 7f29e17fa700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f29c806aed0 0x7f29c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.106+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d0020070 con 0x7f29e40cf610 2023-11-24T07:59:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.106+0000 7f29e8a97700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f29c806aed0 0x7f29c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.107+0000 7f29e8a97700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f29c806aed0 0x7f29c806d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f29d8000f60 tx=0x7f29d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.109+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29d0056230 con 0x7f29e40cf610 2023-11-24T07:59:03.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.308+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29e415dab0 con 0x7f29e40cf610 2023-11-24T07:59:03.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.309+0000 7f29e17fa700 1 -- 172.21.15.32:0/2151533143 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29e415dab0 con 0x7f29e40cf610 2023-11-24T07:59:03.311 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:03.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f29c806aed0 msgr2=0x7f29c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f29c806aed0 0x7f29c806d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f29d8000f60 tx=0x7f29d8009040 comp rx=0 tx=0).stop 2023-11-24T07:59:03.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 msgr2=0x7f29e4152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 0x7f29e4152a80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29d0014040 tx=0x7f29d0004750 comp rx=0 tx=0).stop 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 shutdown_connections 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f29c806aed0 0x7f29c806d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29e415f9e0 0x7f29e4152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29e4073f90 0x7f29e4150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 --2- 172.21.15.32:0/2151533143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29e40cf610 0x7f29e4152a80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 >> 172.21.15.32:0/2151533143 conn(0x7f29e40d0bf0 msgr2=0x7f29e415ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 shutdown_connections 2023-11-24T07:59:03.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.311+0000 7f29eacfb700 1 -- 172.21.15.32:0/2151533143 wait complete. 2023-11-24T07:59:03.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.450+0000 7fa4dc253700 1 Processor -- start 2023-11-24T07:59:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.451+0000 7fa4dc253700 1 -- start start 2023-11-24T07:59:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.451+0000 7fa4dc253700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4d40ceb40 0x7fa4d406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.451+0000 7fa4dc253700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d406ba50 con 0x7fa4d415e080 2023-11-24T07:59:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.451+0000 7fa4dc253700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d406b120 con 0x7fa4d40b4310 2023-11-24T07:59:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.451+0000 7fa4dc253700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d40d1200 con 0x7fa4d40ceb40 2023-11-24T07:59:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d9fef700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa4d415e080 0x7fa4d40d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58968/0 (socket says 172.21.15.32:58968) 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d9fef700 1 -- 172.21.15.32:0/3129501129 learned_addr learned my addr 172.21.15.32:0/3129501129 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4da7f0700 1 --2- 172.21.15.32:0/3129501129 >> v2:172.21.15.123:3300/0 conn(0x7fa4d40ceb40 0x7fa4d406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d8fed700 1 -- 172.21.15.32:0/3129501129 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820245938 0 0) 0x7fa4d406ba50 con 0x7fa4d415e080 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d8fed700 1 -- 172.21.15.32:0/3129501129 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4c00035b0 con 0x7fa4d415e080 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4da7f0700 1 -- 172.21.15.32:0/3129501129 >> v1:172.21.15.123:6789/0 conn(0x7fa4d40b4310 legacy=0x7fa4d40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d8fed700 1 -- 172.21.15.32:0/3129501129 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781824274 0 0) 0x7fa4d406b120 con 0x7fa4d40b4310 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4da7f0700 1 -- 172.21.15.32:0/3129501129 >> v1:172.21.15.32:6789/0 conn(0x7fa4d415e080 legacy=0x7fa4d40d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4da7f0700 1 -- 172.21.15.32:0/3129501129 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d406b400 con 0x7fa4d40ceb40 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4da7f0700 1 --2- 172.21.15.32:0/3129501129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d406bee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009d90 tx=0x7fa4cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88aa66a433c9ef22 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d8fed700 1 -- 172.21.15.32:0/3129501129 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00e070 con 0x7fa4d40ceb40 2023-11-24T07:59:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d8fed700 1 -- 172.21.15.32:0/3129501129 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc00bb20 con 0x7fa4d40ceb40 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.452+0000 7fa4d8fed700 1 -- 172.21.15.32:0/3129501129 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00bed0 con 0x7fa4d40ceb40 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 -- 172.21.15.32:0/3129501129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 msgr2=0x7fa4d406bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 --2- 172.21.15.32:0/3129501129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d406bee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009d90 tx=0x7fa4cc00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 -- 172.21.15.32:0/3129501129 shutdown_connections 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 --2- 172.21.15.32:0/3129501129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d406bee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 -- 172.21.15.32:0/3129501129 >> 172.21.15.32:0/3129501129 conn(0x7fa4d40b3780 msgr2=0x7fa4d40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 -- 172.21.15.32:0/3129501129 shutdown_connections 2023-11-24T07:59:03.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.453+0000 7fa4dc253700 1 -- 172.21.15.32:0/3129501129 wait complete. 2023-11-24T07:59:03.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.454+0000 7fa4dc253700 1 Processor -- start 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.454+0000 7fa4dc253700 1 -- start start 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4dc253700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4d40b4310 0x7fa4d41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4dc253700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4dc253700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 0x7fa4d40b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4dc253700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d40b77e0 con 0x7fa4d415e080 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4dc253700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d415a550 con 0x7fa4d40ceb40 2023-11-24T07:59:03.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4dc253700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d40c6750 con 0x7fa4d40b4310 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 0x7fa4d40b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 0x7fa4d40b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57266/0 (socket says 172.21.15.32:57266) 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 -- 172.21.15.32:0/2907979149 learned_addr learned my addr 172.21.15.32:0/2907979149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4d9fef700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4d40b4310 0x7fa4d41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4d97ee700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 -- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4d40b4310 msgr2=0x7fa4d41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4d40b4310 0x7fa4d41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 -- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 msgr2=0x7fa4d415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.455+0000 7fa4da7f0700 1 -- 172.21.15.32:0/2907979149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc009a70 con 0x7fa4d415e080 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.456+0000 7fa4da7f0700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 0x7fa4d40b72a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc013000 tx=0x7fa4cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.456+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc01b810 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.456+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc020070 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.456+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d40c6a30 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.456+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc01bbb0 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.456+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d40c6ff0 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.457+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d4068fb0 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.457+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4cc01bd10 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.458+0000 7fa4ca7fc700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa4b806af00 0x7fa4b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.458+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4cc022070 con 0x7fa4d415e080 2023-11-24T07:59:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.458+0000 7fa4d9fef700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa4b806af00 0x7fa4b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.458+0000 7fa4d9fef700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa4b806af00 0x7fa4b806d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4005910 tx=0x7fa4c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.462+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4cc05a4a0 con 0x7fa4d415e080 2023-11-24T07:59:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.663+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4d406c330 con 0x7fa4d415e080 2023-11-24T07:59:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.664+0000 7fa4ca7fc700 1 -- 172.21.15.32:0/2907979149 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4cc02c020 con 0x7fa4d415e080 2023-11-24T07:59:03.665 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa4b806af00 msgr2=0x7fa4b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa4b806af00 0x7fa4b806d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4005910 tx=0x7fa4c4009040 comp rx=0 tx=0).stop 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 msgr2=0x7fa4d40b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 0x7fa4d40b72a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc013000 tx=0x7fa4cc004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 shutdown_connections 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa4b806af00 0x7fa4b806d3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4d415e080 0x7fa4d40b72a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4d40b4310 0x7fa4d41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 --2- 172.21.15.32:0/2907979149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4d40ceb40 0x7fa4d415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 >> 172.21.15.32:0/2907979149 conn(0x7fa4d40b3780 msgr2=0x7fa4d406ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 shutdown_connections 2023-11-24T07:59:03.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.666+0000 7fa4dc253700 1 -- 172.21.15.32:0/2907979149 wait complete. 2023-11-24T07:59:03.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:03.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.803+0000 7ffb5afae700 1 Processor -- start 2023-11-24T07:59:03.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.803+0000 7ffb5afae700 1 -- start start 2023-11-24T07:59:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb5afae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb54072df0 0x7ffb540af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb5afae700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540adfd0 con 0x7ffb540622a0 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb5afae700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540ae2b0 con 0x7ffb5415c140 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb5afae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb540ae590 con 0x7ffb54072df0 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb5954b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb540622a0 0x7ffb540af5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58986/0 (socket says 172.21.15.32:58986) 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb5954b700 1 -- 172.21.15.32:0/3403119464 learned_addr learned my addr 172.21.15.32:0/3403119464 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.804+0000 7ffb53fff700 1 --2- 172.21.15.32:0/3403119464 >> v2:172.21.15.123:3300/0 conn(0x7ffb54072df0 0x7ffb540af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299846528 0 0) 0x7ffb540adfd0 con 0x7ffb540622a0 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb440035b0 con 0x7ffb540622a0 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3779620612 0 0) 0x7ffb440035b0 con 0x7ffb540622a0 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 >> v2:172.21.15.123:3300/0 conn(0x7ffb54072df0 msgr2=0x7ffb540af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 --2- 172.21.15.32:0/3403119464 >> v2:172.21.15.123:3300/0 conn(0x7ffb54072df0 0x7ffb540af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 >> v1:172.21.15.123:6789/0 conn(0x7ffb5415c140 legacy=0x7ffb540726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540ae870 con 0x7ffb540622a0 2023-11-24T07:59:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7ffb4c0019f0 con 0x7ffb540622a0 2023-11-24T07:59:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 >> v1:172.21.15.32:6789/0 conn(0x7ffb540622a0 legacy=0x7ffb540af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb44003a00 0x7ffb44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44004850 0x7ffb44004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb44005210 0x7ffb4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540ae590 con 0x7ffb44003a00 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4400c0b0 con 0x7ffb44005210 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4400c270 con 0x7ffb44004850 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb53fff700 1 --2- 172.21.15.32:0/3403119464 >> v2:172.21.15.123:3300/0 conn(0x7ffb54072df0 0x7ffb540af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb44003a00 0x7ffb44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb5954b700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb44005210 0x7ffb4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb53fff700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44004850 0x7ffb44004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 -- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44004850 msgr2=0x7ffb44004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44004850 0x7ffb44004cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 -- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb44005210 msgr2=0x7ffb4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb44005210 0x7ffb4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 -- 172.21.15.32:0/3403119464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440035b0 con 0x7ffb44003a00 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb5954b700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb44005210 0x7ffb4400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb5954b700 1 -- 172.21.15.32:0/3403119464 reap_dead start 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.805+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb44003a00 0x7ffb44003e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffb480017c0 tx=0x7ffb44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1699b616b09cc07c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480052d0 con 0x7ffb44003a00 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48005450 con 0x7ffb44003a00 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb537fe700 1 -- 172.21.15.32:0/3403119464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480038c0 con 0x7ffb44003a00 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 -- 172.21.15.32:0/3403119464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb44003a00 msgr2=0x7ffb44003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb44003a00 0x7ffb44003e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffb480017c0 tx=0x7ffb44005880 comp rx=0 tx=0).stop 2023-11-24T07:59:03.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 -- 172.21.15.32:0/3403119464 shutdown_connections 2023-11-24T07:59:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 --2- 172.21.15.32:0/3403119464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb44003a00 0x7ffb44003e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 -- 172.21.15.32:0/3403119464 >> 172.21.15.32:0/3403119464 conn(0x7ffb540b57b0 msgr2=0x7ffb540b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 -- 172.21.15.32:0/3403119464 shutdown_connections 2023-11-24T07:59:03.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.806+0000 7ffb5afae700 1 -- 172.21.15.32:0/3403119464 wait complete. 2023-11-24T07:59:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.807+0000 7ffb5afae700 1 Processor -- start 2023-11-24T07:59:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 -- start start 2023-11-24T07:59:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44003a00 0x7ffb5414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb54150b80 0x7ffb541f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 0x7ffb5415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5415a860 con 0x7ffb54159e70 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5414ecd0 con 0x7ffb54150b80 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5afae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb541f21d0 con 0x7ffb44003a00 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5954b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 0x7ffb5415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.808+0000 7ffb5954b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 0x7ffb5415a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57286/0 (socket says 172.21.15.32:57286) 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 -- 172.21.15.32:0/89093710 learned_addr learned my addr 172.21.15.32:0/89093710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:03.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44003a00 0x7ffb5414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb53fff700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb54150b80 0x7ffb541f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 -- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44003a00 msgr2=0x7ffb5414e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44003a00 0x7ffb5414e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 -- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb54150b80 msgr2=0x7ffb541f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb54150b80 0x7ffb541f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 -- 172.21.15.32:0/89093710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb541f24b0 con 0x7ffb54159e70 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5954b700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 0x7ffb5415a320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb540b3020 tx=0x7ffb4c00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00ba90 con 0x7ffb54159e70 2023-11-24T07:59:03.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c00bbf0 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb541f2790 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007030 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.809+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb540cbff0 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.810+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb540cc1b0 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.811+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb4c007190 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.811+0000 7ffb517fa700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ffb3806aed0 0x7ffb3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.811+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb4c0896c0 con 0x7ffb54159e70 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.811+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ffb3806aed0 0x7ffb3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.812+0000 7ffb58d4a700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ffb3806aed0 0x7ffb3806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffb48005040 tx=0x7ffb48009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:03.816+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb4c054770 con 0x7ffb54159e70 2023-11-24T07:59:04.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.017+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb54067f60 con 0x7ffb54159e70 2023-11-24T07:59:04.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.018+0000 7ffb517fa700 1 -- 172.21.15.32:0/89093710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7ffb4c019020 con 0x7ffb54159e70 2023-11-24T07:59:04.020 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:04.020 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ffb3806aed0 msgr2=0x7ffb3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ffb3806aed0 0x7ffb3806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffb48005040 tx=0x7ffb48009030 comp rx=0 tx=0).stop 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 msgr2=0x7ffb5415a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 0x7ffb5415a320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb540b3020 tx=0x7ffb4c00df10 comp rx=0 tx=0).stop 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 shutdown_connections 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ffb3806aed0 0x7ffb3806d390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb54159e70 0x7ffb5415a320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb44003a00 0x7ffb5414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 --2- 172.21.15.32:0/89093710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb54150b80 0x7ffb541f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 >> 172.21.15.32:0/89093710 conn(0x7ffb540b57b0 msgr2=0x7ffb5415b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 shutdown_connections 2023-11-24T07:59:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.020+0000 7ffb5afae700 1 -- 172.21.15.32:0/89093710 wait complete. 2023-11-24T07:59:04.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:04.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:04.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.160+0000 7f6746da3700 1 Processor -- start 2023-11-24T07:59:04.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.160+0000 7f6746da3700 1 -- start start 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6746da3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67400c6c30 0x7f67400cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6746da3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f67400cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6746da3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67400d0140 con 0x7f674015bf50 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6746da3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674015c330 con 0x7f67400c6c30 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6746da3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67400cf3d0 con 0x7f67400cef40 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6744b3f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f674015bf50 0x7f67400ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59002/0 (socket says 172.21.15.32:59002) 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6744b3f700 1 -- 172.21.15.32:0/4129593653 learned_addr learned my addr 172.21.15.32:0/4129593653 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f673ffff700 1 --2- 172.21.15.32:0/4129593653 >> v2:172.21.15.123:3300/0 conn(0x7f67400c6c30 0x7f67400cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f6745340700 1 --2- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f67400cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f673f7fe700 1 -- 172.21.15.32:0/4129593653 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670453001 0 0) 0x7f67400d0140 con 0x7f674015bf50 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.161+0000 7f673f7fe700 1 -- 172.21.15.32:0/4129593653 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67300035b0 con 0x7f674015bf50 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673ffff700 1 -- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 msgr2=0x7f67400cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673ffff700 1 --2- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f67400cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673ffff700 1 -- 172.21.15.32:0/4129593653 >> v1:172.21.15.32:6789/0 conn(0x7f674015bf50 legacy=0x7f67400ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673ffff700 1 -- 172.21.15.32:0/4129593653 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400d03e0 con 0x7f67400c6c30 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673ffff700 1 --2- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f67400cea00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f672c009840 tx=0x7f672c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1bd696b2db4ed06 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673f7fe700 1 -- 172.21.15.32:0/4129593653 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c011830 con 0x7f67400c6c30 2023-11-24T07:59:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673f7fe700 1 -- 172.21.15.32:0/4129593653 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c011990 con 0x7f67400c6c30 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f673f7fe700 1 -- 172.21.15.32:0/4129593653 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c011d20 con 0x7f67400c6c30 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f6746da3700 1 -- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 msgr2=0x7f67400cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f6746da3700 1 --2- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f67400cea00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f672c009840 tx=0x7f672c009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f6746da3700 1 -- 172.21.15.32:0/4129593653 shutdown_connections 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f6746da3700 1 --2- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f67400cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f6746da3700 1 --2- 172.21.15.32:0/4129593653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f67400cea00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.162+0000 7f6746da3700 1 -- 172.21.15.32:0/4129593653 >> 172.21.15.32:0/4129593653 conn(0x7f67400b3780 msgr2=0x7f67400b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.163+0000 7f6746da3700 1 -- 172.21.15.32:0/4129593653 shutdown_connections 2023-11-24T07:59:04.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.163+0000 7f6746da3700 1 -- 172.21.15.32:0/4129593653 wait complete. 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.164+0000 7f6746da3700 1 Processor -- start 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.164+0000 7f6746da3700 1 -- start start 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f674014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f674009c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 0x7f67400ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674009d310 con 0x7f674015bf50 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674014c5a0 con 0x7f67400c6c30 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674009d030 con 0x7f67400cef40 2023-11-24T07:59:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 0x7f67400ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 0x7f67400ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57300/0 (socket says 172.21.15.32:57300) 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 -- 172.21.15.32:0/2471336782 learned_addr learned my addr 172.21.15.32:0/2471336782 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f673ffff700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f674009c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6744b3f700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f674014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 -- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 msgr2=0x7f674009c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f674009c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 -- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 msgr2=0x7f674014c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f674014c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 -- 172.21.15.32:0/2471336782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0094a0 con 0x7f674015bf50 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6745340700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 0x7f67400ccf60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f673800efc0 tx=0x7f673800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738009ea0 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800f040 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.165+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67401f7590 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.166+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800ced0 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.166+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67401f7b50 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.166+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67401f73f0 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.167+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6738014480 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.167+0000 7f673d7fa700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f672406aed0 0x7f672406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.167+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6738089da0 con 0x7f674015bf50 2023-11-24T07:59:04.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.168+0000 7f6744b3f700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f672406aed0 0x7f672406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.168+0000 7f6744b3f700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f672406aed0 0x7f672406d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6734007fd0 tx=0x7f6734007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.172+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67401f73f0 con 0x7f674015bf50 2023-11-24T07:59:04.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.372+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67401f73f0 con 0x7f674015bf50 2023-11-24T07:59:04.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.373+0000 7f673d7fa700 1 -- 172.21.15.32:0/2471336782 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67401f73f0 con 0x7f674015bf50 2023-11-24T07:59:04.375 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:04.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f672406aed0 msgr2=0x7f672406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f672406aed0 0x7f672406d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6734007fd0 tx=0x7f6734007dc0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 msgr2=0x7f67400ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 0x7f67400ccf60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f673800efc0 tx=0x7f673800c3a0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 shutdown_connections 2023-11-24T07:59:04.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f672406aed0 0x7f672406d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674015bf50 0x7f67400ccf60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67400c6c30 0x7f674014c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 --2- 172.21.15.32:0/2471336782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67400cef40 0x7f674009c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 >> 172.21.15.32:0/2471336782 conn(0x7f67400b3780 msgr2=0x7f67400b6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 shutdown_connections 2023-11-24T07:59:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.375+0000 7f6746da3700 1 -- 172.21.15.32:0/2471336782 wait complete. 2023-11-24T07:59:04.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:04.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.515+0000 7f4d795aa700 1 Processor -- start 2023-11-24T07:59:04.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.515+0000 7f4d795aa700 1 -- start start 2023-11-24T07:59:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d795aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d740d39d0 0x7f4d740b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d795aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d740d2930 con 0x7f4d7415e140 2023-11-24T07:59:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d795aa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d740d2c10 con 0x7f4d740d45d0 2023-11-24T07:59:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d795aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7415e520 con 0x7f4d740d39d0 2023-11-24T07:59:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d740d39d0 0x7f4d740b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d740d39d0 0x7f4d740b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57308/0 (socket says 172.21.15.32:57308) 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 -- 172.21.15.32:0/3039304570 learned_addr learned my addr 172.21.15.32:0/3039304570 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 -- 172.21.15.32:0/3039304570 >> v1:172.21.15.123:6789/0 conn(0x7f4d740d45d0 legacy=0x7f4d740b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 -- 172.21.15.32:0/3039304570 >> v1:172.21.15.32:6789/0 conn(0x7f4d7415e140 legacy=0x7f4d740d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 -- 172.21.15.32:0/3039304570 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740d2ef0 con 0x7f4d740d39d0 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d737fe700 1 --2- 172.21.15.32:0/3039304570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d39d0 0x7f4d740b77e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e50 tx=0x7f4d6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7943a074a8e1ab3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.516+0000 7f4d71ffb700 1 -- 172.21.15.32:0/3039304570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00e070 con 0x7f4d740d39d0 2023-11-24T07:59:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d71ffb700 1 -- 172.21.15.32:0/3039304570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c00b9c0 con 0x7f4d740d39d0 2023-11-24T07:59:04.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d71ffb700 1 -- 172.21.15.32:0/3039304570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00bd70 con 0x7f4d740d39d0 2023-11-24T07:59:04.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 -- 172.21.15.32:0/3039304570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d39d0 msgr2=0x7f4d740b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 --2- 172.21.15.32:0/3039304570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d39d0 0x7f4d740b77e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e50 tx=0x7f4d6c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 -- 172.21.15.32:0/3039304570 shutdown_connections 2023-11-24T07:59:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 --2- 172.21.15.32:0/3039304570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d39d0 0x7f4d740b77e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 -- 172.21.15.32:0/3039304570 >> 172.21.15.32:0/3039304570 conn(0x7f4d7406b410 msgr2=0x7f4d7406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 -- 172.21.15.32:0/3039304570 shutdown_connections 2023-11-24T07:59:04.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.517+0000 7f4d795aa700 1 -- 172.21.15.32:0/3039304570 wait complete. 2023-11-24T07:59:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.518+0000 7f4d795aa700 1 Processor -- start 2023-11-24T07:59:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 -- start start 2023-11-24T07:59:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d740d39d0 0x7f4d740c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 0x7f4d7414b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d7415e140 0x7f4d740b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740b4ea0 con 0x7f4d740d45d0 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740b5180 con 0x7f4d740d39d0 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d795aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740b5460 con 0x7f4d7415e140 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 0x7f4d7414b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 0x7f4d7414b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57312/0 (socket says 172.21.15.32:57312) 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.519+0000 7f4d727fc700 1 -- 172.21.15.32:0/2893833888 learned_addr learned my addr 172.21.15.32:0/2893833888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d72ffd700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d740d39d0 0x7f4d740c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d737fe700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d7415e140 0x7f4d740b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d727fc700 1 -- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d7415e140 msgr2=0x7f4d740b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d727fc700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d7415e140 0x7f4d740b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d727fc700 1 -- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d740d39d0 msgr2=0x7f4d740c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d727fc700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d740d39d0 0x7f4d740c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d727fc700 1 -- 172.21.15.32:0/2893833888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009b30 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d727fc700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 0x7f4d7414b340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c4e0 tx=0x7f4d6400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d640052d0 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d64005430 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d740b5740 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d640038c0 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.520+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d740b5d00 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.521+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740b5ec0 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.522+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d64003a20 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.522+0000 7f4d5b7fe700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4d6006aed0 0x7f4d6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.522+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d64088b40 con 0x7f4d740d45d0 2023-11-24T07:59:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.522+0000 7f4d72ffd700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4d6006aed0 0x7f4d6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.523+0000 7f4d72ffd700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4d6006aed0 0x7f4d6006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000f60 tx=0x7f4d68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.527+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d64053b60 con 0x7f4d740d45d0 2023-11-24T07:59:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.730+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d740d1bb0 con 0x7f4d740d45d0 2023-11-24T07:59:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.730+0000 7f4d5b7fe700 1 -- 172.21.15.32:0/2893833888 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d640170a0 con 0x7f4d740d45d0 2023-11-24T07:59:04.732 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4d6006aed0 msgr2=0x7f4d6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4d6006aed0 0x7f4d6006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d68000f60 tx=0x7f4d68009040 comp rx=0 tx=0).stop 2023-11-24T07:59:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 msgr2=0x7f4d7414b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 0x7f4d7414b340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c4e0 tx=0x7f4d6400c8a0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 shutdown_connections 2023-11-24T07:59:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4d6006aed0 0x7f4d6006d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4d7415e140 0x7f4d740b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d740d39d0 0x7f4d740c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 --2- 172.21.15.32:0/2893833888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d740d45d0 0x7f4d7414b340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 >> 172.21.15.32:0/2893833888 conn(0x7f4d7406b410 msgr2=0x7f4d740d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 shutdown_connections 2023-11-24T07:59:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.732+0000 7f4d795aa700 1 -- 172.21.15.32:0/2893833888 wait complete. 2023-11-24T07:59:04.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.873+0000 7f0ac3554700 1 Processor -- start 2023-11-24T07:59:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.873+0000 7f0ac3554700 1 -- start start 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac3554700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc15bf50 0x7f0abc0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac3554700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0abc0c6c30 0x7f0abc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac3554700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc0d0140 con 0x7f0abc0ce830 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac3554700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc15c330 con 0x7f0abc0c6c30 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac3554700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0cf3c0 con 0x7f0abc15bf50 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0abc0c6c30 0x7f0abc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0abc0c6c30 0x7f0abc0cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57322/0 (socket says 172.21.15.32:57322) 2023-11-24T07:59:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 -- 172.21.15.32:0/266144006 learned_addr learned my addr 172.21.15.32:0/266144006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac12f0700 1 --2- 172.21.15.32:0/266144006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc15bf50 0x7f0abc0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 -- 172.21.15.32:0/266144006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc15bf50 msgr2=0x7f0abc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 --2- 172.21.15.32:0/266144006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc15bf50 0x7f0abc0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 -- 172.21.15.32:0/266144006 >> v1:172.21.15.32:6789/0 conn(0x7f0abc0ce830 legacy=0x7f0abc0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 -- 172.21.15.32:0/266144006 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0d03e0 con 0x7f0abc0c6c30 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ac1af1700 1 --2- 172.21.15.32:0/266144006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0c6c30 0x7f0abc0cc6c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009e50 tx=0x7f0ab400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87f6d24e9b235b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ab3fff700 1 -- 172.21.15.32:0/266144006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400e070 con 0x7f0abc0c6c30 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ab3fff700 1 -- 172.21.15.32:0/266144006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab400b9c0 con 0x7f0abc0c6c30 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.874+0000 7f0ab3fff700 1 -- 172.21.15.32:0/266144006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400bd70 con 0x7f0abc0c6c30 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 -- 172.21.15.32:0/266144006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0c6c30 msgr2=0x7f0abc0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 --2- 172.21.15.32:0/266144006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0c6c30 0x7f0abc0cc6c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009e50 tx=0x7f0ab400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 -- 172.21.15.32:0/266144006 shutdown_connections 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 --2- 172.21.15.32:0/266144006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0c6c30 0x7f0abc0cc6c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 --2- 172.21.15.32:0/266144006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc15bf50 0x7f0abc0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 -- 172.21.15.32:0/266144006 >> 172.21.15.32:0/266144006 conn(0x7f0abc0b3780 msgr2=0x7f0abc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 -- 172.21.15.32:0/266144006 shutdown_connections 2023-11-24T07:59:04.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.875+0000 7f0ac3554700 1 -- 172.21.15.32:0/266144006 wait complete. 2023-11-24T07:59:04.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.876+0000 7f0ac3554700 1 Processor -- start 2023-11-24T07:59:04.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.876+0000 7f0ac3554700 1 -- start start 2023-11-24T07:59:04.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac3554700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc0c6c30 0x7f0abc06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac3554700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 0x7f0abc14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac3554700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0abc15bf50 0x7f0abc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac3554700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc06cfb0 con 0x7f0abc0ce830 2023-11-24T07:59:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac3554700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc06b9d0 con 0x7f0abc15bf50 2023-11-24T07:59:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac3554700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc06bcb0 con 0x7f0abc0c6c30 2023-11-24T07:59:04.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 0x7f0abc14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 0x7f0abc14bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57326/0 (socket says 172.21.15.32:57326) 2023-11-24T07:59:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 -- 172.21.15.32:0/1917050520 learned_addr learned my addr 172.21.15.32:0/1917050520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac1af1700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0abc15bf50 0x7f0abc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac12f0700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc0c6c30 0x7f0abc06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 -- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc0c6c30 msgr2=0x7f0abc06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc0c6c30 0x7f0abc06ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 -- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0abc15bf50 msgr2=0x7f0abc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0abc15bf50 0x7f0abc06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 -- 172.21.15.32:0/1917050520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009b30 con 0x7f0abc0ce830 2023-11-24T07:59:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac1af1700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0abc15bf50 0x7f0abc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac12f0700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc0c6c30 0x7f0abc06ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.877+0000 7f0ac0aef700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 0x7f0abc14bf80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00c4e0 tx=0x7f0aac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.878+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0052d0 con 0x7f0abc0ce830 2023-11-24T07:59:04.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.878+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac005430 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.878+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0038c0 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.878+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc06bf90 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.878+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc06c550 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.879+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc06c710 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.879+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0aac003a20 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.880+0000 7f0ab1ffb700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0aa006aed0 0x7f0aa006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.880+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aac088a20 con 0x7f0abc0ce830 2023-11-24T07:59:04.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.880+0000 7f0ac12f0700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0aa006aed0 0x7f0aa006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.880+0000 7f0ac12f0700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0aa006aed0 0x7f0aa006d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009820 tx=0x7f0ab8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:04.884+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0aac053b60 con 0x7f0abc0ce830 2023-11-24T07:59:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.086+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0abc068fb0 con 0x7f0abc0ce830 2023-11-24T07:59:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.086+0000 7f0ab1ffb700 1 -- 172.21.15.32:0/1917050520 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0aac0170a0 con 0x7f0abc0ce830 2023-11-24T07:59:05.088 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0aa006aed0 msgr2=0x7f0aa006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0aa006aed0 0x7f0aa006d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009820 tx=0x7f0ab8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 msgr2=0x7f0abc14bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 0x7f0abc14bf80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00c4e0 tx=0x7f0aac00c8a0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 shutdown_connections 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0aa006aed0 0x7f0aa006d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0abc15bf50 0x7f0abc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0abc0c6c30 0x7f0abc06ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 --2- 172.21.15.32:0/1917050520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0abc0ce830 0x7f0abc14bf80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 >> 172.21.15.32:0/1917050520 conn(0x7f0abc0b3780 msgr2=0x7f0abc0c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 shutdown_connections 2023-11-24T07:59:05.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.089+0000 7f0ac3554700 1 -- 172.21.15.32:0/1917050520 wait complete. 2023-11-24T07:59:05.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:05.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.225+0000 7fea294c6700 1 Processor -- start 2023-11-24T07:59:05.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.225+0000 7fea294c6700 1 -- start start 2023-11-24T07:59:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.226+0000 7fea294c6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea2406c710 0x7fea2415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.226+0000 7fea294c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2415f520 con 0x7fea2415ea60 2023-11-24T07:59:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.226+0000 7fea294c6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2415f800 con 0x7fea24144660 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.226+0000 7fea294c6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240b4230 con 0x7fea2406c710 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.226+0000 7fea227fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea2415ea60 0x7fea2406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59044/0 (socket says 172.21.15.32:59044) 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.226+0000 7fea227fc700 1 -- 172.21.15.32:0/4219532919 learned_addr learned my addr 172.21.15.32:0/4219532919 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea237fe700 1 --2- 172.21.15.32:0/4219532919 >> v2:172.21.15.123:3300/0 conn(0x7fea2406c710 0x7fea2415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512100887 0 0) 0x7fea2415f520 con 0x7fea2415ea60 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea100035b0 con 0x7fea2415ea60 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3792648923 0 0) 0x7fea100035b0 con 0x7fea2415ea60 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 >> v2:172.21.15.123:3300/0 conn(0x7fea2406c710 msgr2=0x7fea2415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 --2- 172.21.15.32:0/4219532919 >> v2:172.21.15.123:3300/0 conn(0x7fea2406c710 0x7fea2415eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 >> v1:172.21.15.123:6789/0 conn(0x7fea24144660 legacy=0x7fea240b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea240b3e50 con 0x7fea2415ea60 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fea0c001670 con 0x7fea2415ea60 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 >> v1:172.21.15.32:6789/0 conn(0x7fea2415ea60 legacy=0x7fea2406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10003a00 0x7fea10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea10004850 0x7fea10004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10005210 0x7fea1000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea240b4230 con 0x7fea10003a00 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1000c1b0 con 0x7fea10005210 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1000c370 con 0x7fea10004850 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea22ffd700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10003a00 0x7fea10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea227fc700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea10004850 0x7fea10004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.227+0000 7fea237fe700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10005210 0x7fea1000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea22ffd700 1 -- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea10004850 msgr2=0x7fea10004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea22ffd700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea10004850 0x7fea10004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea22ffd700 1 -- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10005210 msgr2=0x7fea1000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea22ffd700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10005210 0x7fea1000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea237fe700 1 -- 172.21.15.32:0/4219532919 reap_dead start 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea22ffd700 1 -- 172.21.15.32:0/4219532919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea100035b0 con 0x7fea10003a00 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea22ffd700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10003a00 0x7fea10003e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea140017c0 tx=0x7fea10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc7fde7d2c080a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140052d0 con 0x7fea10003a00 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14005450 con 0x7fea10003a00 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea21ffb700 1 -- 172.21.15.32:0/4219532919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140038c0 con 0x7fea10003a00 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea294c6700 1 -- 172.21.15.32:0/4219532919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10003a00 msgr2=0x7fea10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea294c6700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10003a00 0x7fea10003e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea140017c0 tx=0x7fea10005880 comp rx=0 tx=0).stop 2023-11-24T07:59:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea294c6700 1 -- 172.21.15.32:0/4219532919 shutdown_connections 2023-11-24T07:59:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea294c6700 1 --2- 172.21.15.32:0/4219532919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea10003a00 0x7fea10003e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea294c6700 1 -- 172.21.15.32:0/4219532919 >> 172.21.15.32:0/4219532919 conn(0x7fea24071ac0 msgr2=0x7fea24071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.228+0000 7fea294c6700 1 -- 172.21.15.32:0/4219532919 shutdown_connections 2023-11-24T07:59:05.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.229+0000 7fea294c6700 1 -- 172.21.15.32:0/4219532919 wait complete. 2023-11-24T07:59:05.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.230+0000 7fea294c6700 1 Processor -- start 2023-11-24T07:59:05.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.230+0000 7fea294c6700 1 -- start start 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10003a00 0x7fea2415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea2415bd40 0x7fea2413b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 0x7fea241382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2413ecb0 con 0x7fea2415cc30 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2415d4b0 con 0x7fea10003a00 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2413ef00 con 0x7fea2415bd40 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 0x7fea241382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 0x7fea241382d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57348/0 (socket says 172.21.15.32:57348) 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 -- 172.21.15.32:0/991891556 learned_addr learned my addr 172.21.15.32:0/991891556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea22ffd700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10003a00 0x7fea2415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea227fc700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea2415bd40 0x7fea2413b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 -- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea2415bd40 msgr2=0x7fea2413b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea2415bd40 0x7fea2413b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 -- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10003a00 msgr2=0x7fea2415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10003a00 0x7fea2415a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 -- 172.21.15.32:0/991891556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2413f1e0 con 0x7fea2415cc30 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea237fe700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 0x7fea241382d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea2406d400 tx=0x7fea1c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00e040 con 0x7fea2415cc30 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00f040 con 0x7fea2415cc30 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00fbe0 con 0x7fea2415cc30 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2413f4c0 con 0x7fea2415cc30 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.231+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea24138810 con 0x7fea2415cc30 2023-11-24T07:59:05.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.232+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2413f830 con 0x7fea2415cc30 2023-11-24T07:59:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.233+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea1c00fd40 con 0x7fea2415cc30 2023-11-24T07:59:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.233+0000 7fea1b7fe700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fea0406aed0 0x7fea0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.233+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea1c089a80 con 0x7fea2415cc30 2023-11-24T07:59:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.233+0000 7fea22ffd700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fea0406aed0 0x7fea0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.234+0000 7fea22ffd700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fea0406aed0 0x7fea0406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea14000c00 tx=0x7fea14009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.238+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea1c054b20 con 0x7fea2415cc30 2023-11-24T07:59:05.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.440+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea240cd970 con 0x7fea2415cc30 2023-11-24T07:59:05.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.441+0000 7fea1b7fe700 1 -- 172.21.15.32:0/991891556 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fea1c058140 con 0x7fea2415cc30 2023-11-24T07:59:05.443 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:05.443 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fea0406aed0 msgr2=0x7fea0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fea0406aed0 0x7fea0406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea14000c00 tx=0x7fea14009030 comp rx=0 tx=0).stop 2023-11-24T07:59:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 msgr2=0x7fea241382d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 0x7fea241382d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea2406d400 tx=0x7fea1c011ed0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 shutdown_connections 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fea0406aed0 0x7fea0406d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea2415cc30 0x7fea241382d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea10003a00 0x7fea2415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 --2- 172.21.15.32:0/991891556 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea2415bd40 0x7fea2413b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 >> 172.21.15.32:0/991891556 conn(0x7fea24071ac0 msgr2=0x7fea241418e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 shutdown_connections 2023-11-24T07:59:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.443+0000 7fea294c6700 1 -- 172.21.15.32:0/991891556 wait complete. 2023-11-24T07:59:05.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:05.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.586+0000 7fa0e2094700 1 Processor -- start 2023-11-24T07:59:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.586+0000 7fa0e2094700 1 -- start start 2023-11-24T07:59:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0e2094700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0dc0746a0 0x7fa0dc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0e2094700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc0afc90 0x7fa0dc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0e2094700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc0b3c60 con 0x7fa0dc15e160 2023-11-24T07:59:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0e2094700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc0b4500 con 0x7fa0dc0746a0 2023-11-24T07:59:05.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0e2094700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0b58c0 con 0x7fa0dc0afc90 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0daffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0dc0746a0 0x7fa0dc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0db7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0dc15e160 0x7fa0dc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59064/0 (socket says 172.21.15.32:59064) 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0db7fe700 1 -- 172.21.15.32:0/986067975 learned_addr learned my addr 172.21.15.32:0/986067975 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0daffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0dc0746a0 0x7fa0dc0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57354/0 (socket says 172.21.15.32:57354) 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0dbfff700 1 --2- 172.21.15.32:0/986067975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc0afc90 0x7fa0dc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0daffd700 1 -- 172.21.15.32:0/986067975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc0afc90 msgr2=0x7fa0dc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0daffd700 1 --2- 172.21.15.32:0/986067975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc0afc90 0x7fa0dc06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0daffd700 1 -- 172.21.15.32:0/986067975 >> v1:172.21.15.32:6789/0 conn(0x7fa0dc15e160 legacy=0x7fa0dc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0da7fc700 1 -- 172.21.15.32:0/986067975 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024712105 0 0) 0x7fa0dc0b3c60 con 0x7fa0dc15e160 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.587+0000 7fa0daffd700 1 -- 172.21.15.32:0/986067975 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc0b0120 con 0x7fa0dc0746a0 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0daffd700 1 --2- 172.21.15.32:0/986067975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc0b5da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009840 tx=0x7fa0cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26b28e688eb1e0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0da7fc700 1 -- 172.21.15.32:0/986067975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc011830 con 0x7fa0dc0746a0 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0da7fc700 1 -- 172.21.15.32:0/986067975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc011990 con 0x7fa0dc0746a0 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0da7fc700 1 -- 172.21.15.32:0/986067975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc011d20 con 0x7fa0dc0746a0 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 -- 172.21.15.32:0/986067975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 msgr2=0x7fa0dc0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 --2- 172.21.15.32:0/986067975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc0b5da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009840 tx=0x7fa0cc009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 -- 172.21.15.32:0/986067975 shutdown_connections 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 --2- 172.21.15.32:0/986067975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc0afc90 0x7fa0dc06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 --2- 172.21.15.32:0/986067975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc0b5da0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 -- 172.21.15.32:0/986067975 >> 172.21.15.32:0/986067975 conn(0x7fa0dc0d0bf0 msgr2=0x7fa0dc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 -- 172.21.15.32:0/986067975 shutdown_connections 2023-11-24T07:59:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.588+0000 7fa0e2094700 1 -- 172.21.15.32:0/986067975 wait complete. 2023-11-24T07:59:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.590+0000 7fa0e2094700 1 Processor -- start 2023-11-24T07:59:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.590+0000 7fa0e2094700 1 -- start start 2023-11-24T07:59:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0e2094700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0e2094700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0dc0afc90 0x7fa0dc153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0e2094700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc15e160 0x7fa0dc141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0e2094700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc155750 con 0x7fa0dc0746a0 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0e2094700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc154b40 con 0x7fa0dc0afc90 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0e2094700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc154e20 con 0x7fa0dc15e160 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57358/0 (socket says 172.21.15.32:57358) 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 -- 172.21.15.32:0/1335732976 learned_addr learned my addr 172.21.15.32:0/1335732976 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0daffd700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0dc0afc90 0x7fa0dc153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0dbfff700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc15e160 0x7fa0dc141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 -- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc15e160 msgr2=0x7fa0dc141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc15e160 0x7fa0dc141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 -- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0dc0afc90 msgr2=0x7fa0dc153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.591+0000 7fa0db7fe700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0dc0afc90 0x7fa0dc153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0db7fe700 1 -- 172.21.15.32:0/1335732976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0094a0 con 0x7fa0dc0746a0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0dbfff700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc15e160 0x7fa0dc141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0db7fe700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc154560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400c550 tx=0x7fa0c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4004c80 con 0x7fa0dc0746a0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c4004e00 con 0x7fa0dc0746a0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc155100 con 0x7fa0dc0746a0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4007310 con 0x7fa0dc0746a0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.592+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc153e70 con 0x7fa0dc0746a0 2023-11-24T07:59:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.593+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc068fb0 con 0x7fa0dc0746a0 2023-11-24T07:59:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.593+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0c4007470 con 0x7fa0dc0746a0 2023-11-24T07:59:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.594+0000 7fa0d3fff700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0bc06aed0 0x7fa0bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.594+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0c4089220 con 0x7fa0dc0746a0 2023-11-24T07:59:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.594+0000 7fa0daffd700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0bc06aed0 0x7fa0bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.595+0000 7fa0daffd700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0bc06aed0 0x7fa0bc06d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0090b0 tx=0x7fa0cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.599+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0c40542d0 con 0x7fa0dc0746a0 2023-11-24T07:59:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.797+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0dc06c140 con 0x7fa0dc0746a0 2023-11-24T07:59:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.797+0000 7fa0d3fff700 1 -- 172.21.15.32:0/1335732976 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0c40578f0 con 0x7fa0dc0746a0 2023-11-24T07:59:05.799 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.799+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0bc06aed0 msgr2=0x7fa0bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.799+0000 7fa0e2094700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0bc06aed0 0x7fa0bc06d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0090b0 tx=0x7fa0cc008030 comp rx=0 tx=0).stop 2023-11-24T07:59:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 msgr2=0x7fa0dc154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc154560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400c550 tx=0x7fa0c400c860 comp rx=0 tx=0).stop 2023-11-24T07:59:05.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 shutdown_connections 2023-11-24T07:59:05.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0bc06aed0 0x7fa0bc06d390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0dc15e160 0x7fa0dc141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0dc0746a0 0x7fa0dc154560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 --2- 172.21.15.32:0/1335732976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0dc0afc90 0x7fa0dc153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 >> 172.21.15.32:0/1335732976 conn(0x7fa0dc0d0bf0 msgr2=0x7fa0dc0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 shutdown_connections 2023-11-24T07:59:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.800+0000 7fa0e2094700 1 -- 172.21.15.32:0/1335732976 wait complete. 2023-11-24T07:59:05.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.940+0000 7f3869c4e700 1 Processor -- start 2023-11-24T07:59:05.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.940+0000 7f3869c4e700 1 -- start start 2023-11-24T07:59:05.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3869c4e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38640cf610 0x7f38640add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3869c4e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3864073f90 0x7f3864074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3869c4e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386406b080 con 0x7f386415f9e0 2023-11-24T07:59:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3869c4e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3864074960 con 0x7f3864073f90 2023-11-24T07:59:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3869c4e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386415c290 con 0x7f38640cf610 2023-11-24T07:59:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3864073f90 0x7f3864074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3864073f90 0x7f3864074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57370/0 (socket says 172.21.15.32:57370) 2023-11-24T07:59:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 -- 172.21.15.32:0/804772703 learned_addr learned my addr 172.21.15.32:0/804772703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f38637fe700 1 --2- 172.21.15.32:0/804772703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38640cf610 0x7f38640add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 -- 172.21.15.32:0/804772703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38640cf610 msgr2=0x7f38640add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 --2- 172.21.15.32:0/804772703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38640cf610 0x7f38640add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 -- 172.21.15.32:0/804772703 >> v1:172.21.15.32:6789/0 conn(0x7f386415f9e0 legacy=0x7f38640b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.941+0000 7f3863fff700 1 -- 172.21.15.32:0/804772703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640cf9f0 con 0x7f3864073f90 2023-11-24T07:59:05.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3863fff700 1 --2- 172.21.15.32:0/804772703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f3864074420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f385c009e50 tx=0x7f385c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4f7c248581e5f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f38627fc700 1 -- 172.21.15.32:0/804772703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00e070 con 0x7f3864073f90 2023-11-24T07:59:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f38627fc700 1 -- 172.21.15.32:0/804772703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c00b910 con 0x7f3864073f90 2023-11-24T07:59:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f38627fc700 1 -- 172.21.15.32:0/804772703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00bcc0 con 0x7f3864073f90 2023-11-24T07:59:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 -- 172.21.15.32:0/804772703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 msgr2=0x7f3864074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 --2- 172.21.15.32:0/804772703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f3864074420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f385c009e50 tx=0x7f385c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 -- 172.21.15.32:0/804772703 shutdown_connections 2023-11-24T07:59:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 --2- 172.21.15.32:0/804772703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f3864074420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 --2- 172.21.15.32:0/804772703 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38640cf610 0x7f38640add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 -- 172.21.15.32:0/804772703 >> 172.21.15.32:0/804772703 conn(0x7f38640d0bf0 msgr2=0x7f38640d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 -- 172.21.15.32:0/804772703 shutdown_connections 2023-11-24T07:59:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.942+0000 7f3869c4e700 1 -- 172.21.15.32:0/804772703 wait complete. 2023-11-24T07:59:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 Processor -- start 2023-11-24T07:59:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 -- start start 2023-11-24T07:59:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f38641f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38640cf610 0x7f38641401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f386415f9e0 0x7f3864142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38641f16e0 con 0x7f3864073f90 2023-11-24T07:59:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864140730 con 0x7f38640cf610 2023-11-24T07:59:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.944+0000 7f3869c4e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386415eca0 con 0x7f386415f9e0 2023-11-24T07:59:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f38641f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f38641f11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57372/0 (socket says 172.21.15.32:57372) 2023-11-24T07:59:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 -- 172.21.15.32:0/1693242461 learned_addr learned my addr 172.21.15.32:0/1693242461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f3863fff700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f386415f9e0 0x7f3864142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f3862ffd700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38640cf610 0x7f38641401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 -- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f386415f9e0 msgr2=0x7f3864142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f386415f9e0 0x7f3864142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 -- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38640cf610 msgr2=0x7f38641401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38640cf610 0x7f38641401f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 -- 172.21.15.32:0/1693242461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c009b30 con 0x7f3864073f90 2023-11-24T07:59:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f38637fe700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f38641f11a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9b0 tx=0x7f385400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854009710 con 0x7f3864073f90 2023-11-24T07:59:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400c850 con 0x7f3864073f90 2023-11-24T07:59:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3864156430 con 0x7f3864073f90 2023-11-24T07:59:05.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400cbc0 con 0x7f3864073f90 2023-11-24T07:59:05.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.945+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38641569f0 con 0x7f3864073f90 2023-11-24T07:59:05.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.946+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3864156d60 con 0x7f3864073f90 2023-11-24T07:59:05.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.947+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f385400cd20 con 0x7f3864073f90 2023-11-24T07:59:05.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.947+0000 7f384bfff700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f385006aed0 0x7f385006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.947+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3854089760 con 0x7f3864073f90 2023-11-24T07:59:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.947+0000 7f3862ffd700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f385006aed0 0x7f385006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.948+0000 7f3862ffd700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f385006aed0 0x7f385006d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38640d0440 tx=0x7f3858009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:05.952+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3854054790 con 0x7f3864073f90 2023-11-24T07:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.151+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3864156b80 con 0x7f3864073f90 2023-11-24T07:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.152+0000 7f384bfff700 1 -- 172.21.15.32:0/1693242461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3854057db0 con 0x7f3864073f90 2023-11-24T07:59:06.154 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:06.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f385006aed0 msgr2=0x7f385006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f385006aed0 0x7f385006d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38640d0440 tx=0x7f3858009040 comp rx=0 tx=0).stop 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 msgr2=0x7f38641f11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f38641f11a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9b0 tx=0x7f385400ed70 comp rx=0 tx=0).stop 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 shutdown_connections 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f385006aed0 0x7f385006d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f386415f9e0 0x7f3864142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3864073f90 0x7f38641f11a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 --2- 172.21.15.32:0/1693242461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38640cf610 0x7f38641401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 >> 172.21.15.32:0/1693242461 conn(0x7f38640d0bf0 msgr2=0x7f38640b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 shutdown_connections 2023-11-24T07:59:06.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.154+0000 7f3869c4e700 1 -- 172.21.15.32:0/1693242461 wait complete. 2023-11-24T07:59:06.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.295+0000 7f57a5ed7700 1 Processor -- start 2023-11-24T07:59:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.295+0000 7f57a5ed7700 1 -- start start 2023-11-24T07:59:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f57a5ed7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 0x7f57a006fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f57a5ed7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a00703a0 con 0x7f57a0070b40 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f57a5ed7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a0070820 con 0x7f57a00cc580 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f57a5ed7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a006fb80 con 0x7f57a015f500 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57a0070b40 0x7f57a006ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59094/0 (socket says 172.21.15.32:59094) 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579effd700 1 -- 172.21.15.32:0/1158511404 learned_addr learned my addr 172.21.15.32:0/1158511404 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579f7fe700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 0x7f57a006fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555181773 0 0) 0x7f57a00703a0 con 0x7f57a0070b40 2023-11-24T07:59:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57880035b0 con 0x7f57a0070b40 2023-11-24T07:59:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657529063 0 0) 0x7f57880035b0 con 0x7f57a0070b40 2023-11-24T07:59:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 msgr2=0x7f57a006fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 0x7f57a006fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 >> v1:172.21.15.123:6789/0 conn(0x7f57a00cc580 legacy=0x7f57a00b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00d0260 con 0x7f57a0070b40 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f5794001670 con 0x7f57a0070b40 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 >> v1:172.21.15.32:6789/0 conn(0x7f57a0070b40 legacy=0x7f57a006ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.296+0000 7f579e7fc700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 0x7f5788003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5788003f70 0x7f5788004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788004d90 0x7f5788005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a006fb80 con 0x7f5788003f70 2023-11-24T07:59:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578800ba70 con 0x7f5788004d90 2023-11-24T07:59:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578800bc30 con 0x7f57a015f500 2023-11-24T07:59:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5788003f70 0x7f5788004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579f7fe700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 0x7f5788003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579ffff700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788004d90 0x7f5788005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 -- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 msgr2=0x7f5788003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a015f500 0x7f5788003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 -- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788004d90 msgr2=0x7f5788005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788004d90 0x7f5788005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 -- 172.21.15.32:0/1158511404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57880035b0 con 0x7f5788003f70 2023-11-24T07:59:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579effd700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5788003f70 0x7f5788004850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57a0070fc0 tx=0x7f5788005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f96da95d387c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579400cd40 con 0x7f5788003f70 2023-11-24T07:59:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794005960 con 0x7f5788003f70 2023-11-24T07:59:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.297+0000 7f579e7fc700 1 -- 172.21.15.32:0/1158511404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794005d10 con 0x7f5788003f70 2023-11-24T07:59:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f57a5ed7700 1 -- 172.21.15.32:0/1158511404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5788003f70 msgr2=0x7f5788004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/1158511404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5788003f70 0x7f5788004850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57a0070fc0 tx=0x7f5788005880 comp rx=0 tx=0).stop 2023-11-24T07:59:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f579ffff700 1 -- 172.21.15.32:0/1158511404 reap_dead start 2023-11-24T07:59:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f57a5ed7700 1 -- 172.21.15.32:0/1158511404 shutdown_connections 2023-11-24T07:59:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f57a5ed7700 1 -- 172.21.15.32:0/1158511404 >> 172.21.15.32:0/1158511404 conn(0x7f57a00b3710 msgr2=0x7f57a00b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f57a5ed7700 1 -- 172.21.15.32:0/1158511404 shutdown_connections 2023-11-24T07:59:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.298+0000 7f57a5ed7700 1 -- 172.21.15.32:0/1158511404 wait complete. 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.299+0000 7f57a5ed7700 1 Processor -- start 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.300+0000 7f57a5ed7700 1 -- start start 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788003f70 0x7f57a00cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 0x7f57a0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a0159d00 0x7f57a0156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01568d0 con 0x7f57a00c77b0 2023-11-24T07:59:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00c5f40 con 0x7f5788003f70 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015a2c0 con 0x7f57a0159d00 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 0x7f57a0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 0x7f57a0155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57394/0 (socket says 172.21.15.32:57394) 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 -- 172.21.15.32:0/49327855 learned_addr learned my addr 172.21.15.32:0/49327855 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579ffff700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a0159d00 0x7f57a0156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579f7fe700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788003f70 0x7f57a00cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 -- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a0159d00 msgr2=0x7f57a0156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a0159d00 0x7f57a0156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 -- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788003f70 msgr2=0x7f57a00cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788003f70 0x7f57a00cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 -- 172.21.15.32:0/49327855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a015fa60 con 0x7f57a00c77b0 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f579effd700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 0x7f57a0155e50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f578800ba40 tx=0x7f5794001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794004a80 con 0x7f57a00c77b0 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794004be0 con 0x7f57a00c77b0 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940070d0 con 0x7f57a00c77b0 2023-11-24T07:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00c5610 con 0x7f57a00c77b0 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.301+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00c5bd0 con 0x7f57a00c77b0 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.302+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0068fb0 con 0x7f57a00c77b0 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.303+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5794007230 con 0x7f57a00c77b0 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.303+0000 7f5787fff700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f578c06aed0 0x7f578c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.303+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f579408ac60 con 0x7f57a00c77b0 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.303+0000 7f579f7fe700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f578c06aed0 0x7f578c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.304+0000 7f579f7fe700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f578c06aed0 0x7f578c06d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5790009c60 tx=0x7f5790009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.308+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5794055f80 con 0x7f57a00c77b0 2023-11-24T07:59:06.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.509+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57a015c3a0 con 0x7f57a00c77b0 2023-11-24T07:59:06.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.510+0000 7f5787fff700 1 -- 172.21.15.32:0/49327855 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5794007da0 con 0x7f57a00c77b0 2023-11-24T07:59:06.512 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f578c06aed0 msgr2=0x7f578c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f578c06aed0 0x7f578c06d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5790009c60 tx=0x7f5790009210 comp rx=0 tx=0).stop 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 msgr2=0x7f57a0155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 0x7f57a0155e50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f578800ba40 tx=0x7f5794001a00 comp rx=0 tx=0).stop 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 shutdown_connections 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f578c06aed0 0x7f578c06d390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57a0159d00 0x7f57a0156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5788003f70 0x7f57a00cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 --2- 172.21.15.32:0/49327855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a00c77b0 0x7f57a0155e50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 >> 172.21.15.32:0/49327855 conn(0x7f57a00b3710 msgr2=0x7f57a015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 shutdown_connections 2023-11-24T07:59:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.512+0000 7f57a5ed7700 1 -- 172.21.15.32:0/49327855 wait complete. 2023-11-24T07:59:06.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:06.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.651+0000 7fa51e71a700 1 Processor -- start 2023-11-24T07:59:06.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa51e71a700 1 -- start start 2023-11-24T07:59:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa51e71a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa51815be00 0x7fa5180cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa51e71a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5180b00e0 con 0x7fa51815c1e0 2023-11-24T07:59:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa51e71a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5180af200 con 0x7fa5180d0040 2023-11-24T07:59:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa51e71a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5180af4d0 con 0x7fa51815be00 2023-11-24T07:59:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa517fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa51815be00 0x7fa5180cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.652+0000 7fa517fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa51815be00 0x7fa5180cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57400/0 (socket says 172.21.15.32:57400) 2023-11-24T07:59:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa517fff700 1 -- 172.21.15.32:0/3576700900 learned_addr learned my addr 172.21.15.32:0/3576700900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa517fff700 1 -- 172.21.15.32:0/3576700900 >> v1:172.21.15.123:6789/0 conn(0x7fa5180d0040 legacy=0x7fa5180b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa517fff700 1 -- 172.21.15.32:0/3576700900 >> v1:172.21.15.32:6789/0 conn(0x7fa51815c1e0 legacy=0x7fa5180aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa517fff700 1 -- 172.21.15.32:0/3576700900 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5180af7b0 con 0x7fa51815be00 2023-11-24T07:59:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa517fff700 1 --2- 172.21.15.32:0/3576700900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa51815be00 0x7fa5180cc660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa5080098f0 tx=0x7fa508009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65e95763dd5f213 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa516ffd700 1 -- 172.21.15.32:0/3576700900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508004230 con 0x7fa51815be00 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa516ffd700 1 -- 172.21.15.32:0/3576700900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa508004390 con 0x7fa51815be00 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.653+0000 7fa516ffd700 1 -- 172.21.15.32:0/3576700900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080036a0 con 0x7fa51815be00 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 -- 172.21.15.32:0/3576700900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa51815be00 msgr2=0x7fa5180cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 --2- 172.21.15.32:0/3576700900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa51815be00 0x7fa5180cc660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa5080098f0 tx=0x7fa508009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 -- 172.21.15.32:0/3576700900 shutdown_connections 2023-11-24T07:59:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 --2- 172.21.15.32:0/3576700900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa51815be00 0x7fa5180cc660 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 -- 172.21.15.32:0/3576700900 >> 172.21.15.32:0/3576700900 conn(0x7fa51809aa00 msgr2=0x7fa51809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 -- 172.21.15.32:0/3576700900 shutdown_connections 2023-11-24T07:59:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.654+0000 7fa51e71a700 1 -- 172.21.15.32:0/3576700900 wait complete. 2023-11-24T07:59:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.655+0000 7fa51e71a700 1 Processor -- start 2023-11-24T07:59:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.655+0000 7fa51e71a700 1 -- start start 2023-11-24T07:59:06.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51e71a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 0x7fa5180c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51e71a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa51815be00 0x7fa51806c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51e71a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51815c1e0 0x7fa518157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51e71a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180c4eb0 con 0x7fa5180d0040 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51e71a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51806b090 con 0x7fa51815c1e0 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51e71a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51806b370 con 0x7fa51815be00 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 0x7fa5180c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 0x7fa5180c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57404/0 (socket says 172.21.15.32:57404) 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 -- 172.21.15.32:0/872164807 learned_addr learned my addr 172.21.15.32:0/872164807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa5177fe700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa51815be00 0x7fa51806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51ccb7700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51815c1e0 0x7fa518157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 -- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa51815be00 msgr2=0x7fa51806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa51815be00 0x7fa51806c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 -- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51815c1e0 msgr2=0x7fa518157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51815c1e0 0x7fa518157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 -- 172.21.15.32:0/872164807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa508009550 con 0x7fa5180d0040 2023-11-24T07:59:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa51ccb7700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51815c1e0 0x7fa518157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.656+0000 7fa517fff700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 0x7fa5180c4970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa5180cf800 tx=0x7fa508004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.657+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508003dc0 con 0x7fa5180d0040 2023-11-24T07:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.657+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50801f070 con 0x7fa5180d0040 2023-11-24T07:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.657+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51806b650 con 0x7fa5180d0040 2023-11-24T07:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.657+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800b8a0 con 0x7fa5180d0040 2023-11-24T07:59:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.657+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51806bc10 con 0x7fa5180d0040 2023-11-24T07:59:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.658+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51806bf80 con 0x7fa5180d0040 2023-11-24T07:59:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.658+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa508021030 con 0x7fa5180d0040 2023-11-24T07:59:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.658+0000 7fa514ff9700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa50406af00 0x7fa50406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.659+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa50808acd0 con 0x7fa5180d0040 2023-11-24T07:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.659+0000 7fa5177fe700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa50406af00 0x7fa50406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.659+0000 7fa5177fe700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa50406af00 0x7fa50406d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007fd0 tx=0x7fa50c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.663+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa508055e10 con 0x7fa5180d0040 2023-11-24T07:59:06.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.864+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa51806bf80 con 0x7fa5180d0040 2023-11-24T07:59:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.865+0000 7fa514ff9700 1 -- 172.21.15.32:0/872164807 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fa508059430 con 0x7fa5180d0040 2023-11-24T07:59:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:06.867 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa50406af00 msgr2=0x7fa50406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa50406af00 0x7fa50406d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007fd0 tx=0x7fa50c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 msgr2=0x7fa5180c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 0x7fa5180c4970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa5180cf800 tx=0x7fa508004750 comp rx=0 tx=0).stop 2023-11-24T07:59:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 shutdown_connections 2023-11-24T07:59:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa50406af00 0x7fa50406d3c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51815c1e0 0x7fa518157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180d0040 0x7fa5180c4970 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 --2- 172.21.15.32:0/872164807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa51815be00 0x7fa51806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 >> 172.21.15.32:0/872164807 conn(0x7fa51809aa00 msgr2=0x7fa51806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 shutdown_connections 2023-11-24T07:59:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:06.867+0000 7fa51e71a700 1 -- 172.21.15.32:0/872164807 wait complete. 2023-11-24T07:59:06.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:06.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.009+0000 7efef4501700 1 Processor -- start 2023-11-24T07:59:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.009+0000 7efef4501700 1 -- start start 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef4501700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efeec0b4020 0x7efeec073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef4501700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeec0744d0 0x7efeec06d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef4501700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec15bf70 con 0x7efeec0af520 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef4501700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec074960 con 0x7efeec0744d0 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef4501700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec15ee00 con 0x7efeec0b4020 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeec0744d0 0x7efeec06d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeec0744d0 0x7efeec06d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57416/0 (socket says 172.21.15.32:57416) 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 -- 172.21.15.32:0/3211729931 learned_addr learned my addr 172.21.15.32:0/3211729931 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef1a9c700 1 --2- 172.21.15.32:0/3211729931 >> v2:172.21.15.123:3300/0 conn(0x7efeec0b4020 0x7efeec073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 -- 172.21.15.32:0/3211729931 >> v2:172.21.15.123:3300/0 conn(0x7efeec0b4020 msgr2=0x7efeec073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 --2- 172.21.15.32:0/3211729931 >> v2:172.21.15.123:3300/0 conn(0x7efeec0b4020 0x7efeec073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 -- 172.21.15.32:0/3211729931 >> v1:172.21.15.123:6789/0 conn(0x7efeec0af520 legacy=0x7efeec0afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 -- 172.21.15.32:0/3211729931 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0b0240 con 0x7efeec0744d0 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.010+0000 7efef2a9e700 1 --2- 172.21.15.32:0/3211729931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0744d0 0x7efeec06d910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efeec0d1ac0 tx=0x7efee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5596665eb3dd92da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef129b700 1 -- 172.21.15.32:0/3211729931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400e070 con 0x7efeec0744d0 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef129b700 1 -- 172.21.15.32:0/3211729931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400b910 con 0x7efeec0744d0 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef129b700 1 -- 172.21.15.32:0/3211729931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400bcc0 con 0x7efeec0744d0 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 -- 172.21.15.32:0/3211729931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0744d0 msgr2=0x7efeec06d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 --2- 172.21.15.32:0/3211729931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0744d0 0x7efeec06d910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efeec0d1ac0 tx=0x7efee400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 -- 172.21.15.32:0/3211729931 shutdown_connections 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 --2- 172.21.15.32:0/3211729931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0744d0 0x7efeec06d910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 --2- 172.21.15.32:0/3211729931 >> v2:172.21.15.123:3300/0 conn(0x7efeec0b4020 0x7efeec073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 -- 172.21.15.32:0/3211729931 >> 172.21.15.32:0/3211729931 conn(0x7efeec0b28f0 msgr2=0x7efeec0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 -- 172.21.15.32:0/3211729931 shutdown_connections 2023-11-24T07:59:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.011+0000 7efef4501700 1 -- 172.21.15.32:0/3211729931 wait complete. 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.012+0000 7efef4501700 1 Processor -- start 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.012+0000 7efef4501700 1 -- start start 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef4501700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0744d0 0x7efeec14add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef4501700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 0x7efeec0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef4501700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec0b4020 0x7efeec1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef4501700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec159460 con 0x7efeec0af520 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef4501700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec14aa90 con 0x7efeec0b4020 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef4501700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec158850 con 0x7efeec0744d0 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef1a9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 0x7efeec0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef1a9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 0x7efeec0c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57420/0 (socket says 172.21.15.32:57420) 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef1a9c700 1 -- 172.21.15.32:0/107096880 learned_addr learned my addr 172.21.15.32:0/107096880 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef229d700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0744d0 0x7efeec14add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.013+0000 7efef2a9e700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec0b4020 0x7efeec1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef1a9c700 1 -- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0744d0 msgr2=0x7efeec14add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef1a9c700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0744d0 0x7efeec14add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef1a9c700 1 -- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec0b4020 msgr2=0x7efeec1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef1a9c700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec0b4020 0x7efeec1598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef1a9c700 1 -- 172.21.15.32:0/107096880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4009b30 con 0x7efeec0af520 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef1a9c700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 0x7efeec0c6760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e840 tx=0x7efedc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc009590 con 0x7efeec0af520 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc013680 con 0x7efeec0af520 2023-11-24T07:59:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec158b30 con 0x7efeec0af520 2023-11-24T07:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc013a10 con 0x7efeec0af520 2023-11-24T07:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.014+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec1590f0 con 0x7efeec0af520 2023-11-24T07:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.015+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec1595d0 con 0x7efeec0af520 2023-11-24T07:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.015+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efedc013b70 con 0x7efeec0af520 2023-11-24T07:59:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.016+0000 7efee2ffd700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efed006aed0 0x7efed006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.016+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efedc089520 con 0x7efeec0af520 2023-11-24T07:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.016+0000 7efef229d700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efed006aed0 0x7efed006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.017+0000 7efef229d700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efed006aed0 0x7efed006d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efee8000f60 tx=0x7efee8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.021+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efedc0545d0 con 0x7efeec0af520 2023-11-24T07:59:07.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.219+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efeec068fb0 con 0x7efeec0af520 2023-11-24T07:59:07.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.219+0000 7efee2ffd700 1 -- 172.21.15.32:0/107096880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7efedc057bf0 con 0x7efeec0af520 2023-11-24T07:59:07.221 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efed006aed0 msgr2=0x7efed006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efed006aed0 0x7efed006d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efee8000f60 tx=0x7efee8009040 comp rx=0 tx=0).stop 2023-11-24T07:59:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 msgr2=0x7efeec0c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 0x7efeec0c6760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e840 tx=0x7efedc00ec00 comp rx=0 tx=0).stop 2023-11-24T07:59:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 shutdown_connections 2023-11-24T07:59:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7efed006aed0 0x7efed006d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec0b4020 0x7efeec1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0744d0 0x7efeec14add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 --2- 172.21.15.32:0/107096880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeec0af520 0x7efeec0c6760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 >> 172.21.15.32:0/107096880 conn(0x7efeec0b28f0 msgr2=0x7efeec0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 shutdown_connections 2023-11-24T07:59:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.221+0000 7efef4501700 1 -- 172.21.15.32:0/107096880 wait complete. 2023-11-24T07:59:07.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.361+0000 7fd1a1f59700 1 Processor -- start 2023-11-24T07:59:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.361+0000 7fd1a1f59700 1 -- start start 2023-11-24T07:59:07.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd1a1f59700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd19c0b49a0 0x7fd19c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd1a1f59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd19c06e370 0x7fd19c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd1a1f59700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c15e9f0 con 0x7fd19c06e880 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd1a1f59700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c0b52c0 con 0x7fd19c06e370 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd1a1f59700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c15ec90 con 0x7fd19c0b49a0 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd19affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd19c06e370 0x7fd19c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd19affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd19c06e370 0x7fd19c0b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57426/0 (socket says 172.21.15.32:57426) 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd19affd700 1 -- 172.21.15.32:0/334477329 learned_addr learned my addr 172.21.15.32:0/334477329 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/334477329 >> v2:172.21.15.123:3300/0 conn(0x7fd19c0b49a0 0x7fd19c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd19affd700 1 -- 172.21.15.32:0/334477329 >> v2:172.21.15.123:3300/0 conn(0x7fd19c0b49a0 msgr2=0x7fd19c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.362+0000 7fd19affd700 1 --2- 172.21.15.32:0/334477329 >> v2:172.21.15.123:3300/0 conn(0x7fd19c0b49a0 0x7fd19c15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19affd700 1 -- 172.21.15.32:0/334477329 >> v1:172.21.15.123:6789/0 conn(0x7fd19c06e880 legacy=0x7fd19c06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19affd700 1 -- 172.21.15.32:0/334477329 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c15ef70 con 0x7fd19c06e370 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/334477329 >> v2:172.21.15.123:3300/0 conn(0x7fd19c0b49a0 0x7fd19c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19affd700 1 --2- 172.21.15.32:0/334477329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e370 0x7fd19c0b4d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd19c072f40 tx=0x7fd18c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f477b7156d4eb99d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19a7fc700 1 -- 172.21.15.32:0/334477329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00b870 con 0x7fd19c06e370 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19a7fc700 1 -- 172.21.15.32:0/334477329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00b9d0 con 0x7fd19c06e370 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd19a7fc700 1 -- 172.21.15.32:0/334477329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00bd60 con 0x7fd19c06e370 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 -- 172.21.15.32:0/334477329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e370 msgr2=0x7fd19c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/334477329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e370 0x7fd19c0b4d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd19c072f40 tx=0x7fd18c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 -- 172.21.15.32:0/334477329 shutdown_connections 2023-11-24T07:59:07.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/334477329 >> v2:172.21.15.123:3300/0 conn(0x7fd19c0b49a0 0x7fd19c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/334477329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e370 0x7fd19c0b4d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 -- 172.21.15.32:0/334477329 >> 172.21.15.32:0/334477329 conn(0x7fd19c0b28f0 msgr2=0x7fd19c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 -- 172.21.15.32:0/334477329 shutdown_connections 2023-11-24T07:59:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.363+0000 7fd1a1f59700 1 -- 172.21.15.32:0/334477329 wait complete. 2023-11-24T07:59:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.365+0000 7fd1a1f59700 1 Processor -- start 2023-11-24T07:59:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.365+0000 7fd1a1f59700 1 -- start start 2023-11-24T07:59:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.365+0000 7fd1a1f59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c06e370 0x7fd19c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 0x7fd19c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c0b49a0 0x7fd19c0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c142290 con 0x7fd19c06e880 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0cfd10 con 0x7fd19c06e370 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0cf420 con 0x7fd19c0b49a0 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 0x7fd19c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 0x7fd19c0cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57430/0 (socket says 172.21.15.32:57430) 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 -- 172.21.15.32:0/3032992522 learned_addr learned my addr 172.21.15.32:0/3032992522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c06e370 0x7fd19c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19bfff700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c0b49a0 0x7fd19c0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 -- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c0b49a0 msgr2=0x7fd19c0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c0b49a0 0x7fd19c0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 -- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c06e370 msgr2=0x7fd19c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c06e370 0x7fd19c0d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 -- 172.21.15.32:0/3032992522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c009550 con 0x7fd19c06e880 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c06e370 0x7fd19c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19bfff700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c0b49a0 0x7fd19c0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd19affd700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 0x7fd19c0cd990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000c00 tx=0x7fd18c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c004520 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c01e070 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c141fb0 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0038a0 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.366+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c0cf9a0 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.367+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c0b26a0 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.368+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd18c003a00 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.368+0000 7fd193fff700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd17c06aed0 0x7fd17c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.368+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd18c020070 con 0x7fd19c06e880 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.368+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd17c06aed0 0x7fd17c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.369+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd17c06aed0 0x7fd17c06d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1840099f0 tx=0x7fd184008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.373+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd18c055c70 con 0x7fd19c06e880 2023-11-24T07:59:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.573+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd19c0cfe80 con 0x7fd19c06e880 2023-11-24T07:59:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.574+0000 7fd193fff700 1 -- 172.21.15.32:0/3032992522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd18c003cb0 con 0x7fd19c06e880 2023-11-24T07:59:07.576 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd17c06aed0 msgr2=0x7fd17c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd17c06aed0 0x7fd17c06d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1840099f0 tx=0x7fd184008040 comp rx=0 tx=0).stop 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 msgr2=0x7fd19c0cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 0x7fd19c0cd990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000c00 tx=0x7fd18c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 shutdown_connections 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd17c06aed0 0x7fd17c06d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd19c0b49a0 0x7fd19c0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c06e370 0x7fd19c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 --2- 172.21.15.32:0/3032992522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c06e880 0x7fd19c0cd990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 >> 172.21.15.32:0/3032992522 conn(0x7fd19c0b28f0 msgr2=0x7fd19c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 shutdown_connections 2023-11-24T07:59:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.576+0000 7fd1a1f59700 1 -- 172.21.15.32:0/3032992522 wait complete. 2023-11-24T07:59:07.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.717+0000 7f43a3d6a700 1 Processor -- start 2023-11-24T07:59:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.717+0000 7f43a3d6a700 1 -- start start 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a3d6a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f439c0cf4e0 0x7f439c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a3d6a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c06de00 con 0x7f439c140540 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a3d6a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c0d0310 con 0x7f439c0cfe80 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a3d6a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0d0bb0 con 0x7f439c0cf4e0 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a1b06700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f439c140540 0x7f439c140920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59150/0 (socket says 172.21.15.32:59150) 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a1b06700 1 -- 172.21.15.32:0/2557134185 learned_addr learned my addr 172.21.15.32:0/2557134185 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:07.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a1305700 1 --2- 172.21.15.32:0/2557134185 >> v2:172.21.15.123:3300/0 conn(0x7f439c0cf4e0 0x7f439c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541580021 0 0) 0x7f439c06de00 con 0x7f439c140540 2023-11-24T07:59:07.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43880035b0 con 0x7f439c140540 2023-11-24T07:59:07.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.718+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609394835 0 0) 0x7f43880035b0 con 0x7f439c140540 2023-11-24T07:59:07.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 >> v2:172.21.15.123:3300/0 conn(0x7f439c0cf4e0 msgr2=0x7f439c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:07.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 --2- 172.21.15.32:0/2557134185 >> v2:172.21.15.123:3300/0 conn(0x7f439c0cf4e0 0x7f439c0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 >> v1:172.21.15.123:6789/0 conn(0x7f439c0cfe80 legacy=0x7f439c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0d05f0 con 0x7f439c140540 2023-11-24T07:59:07.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f4398001670 con 0x7f439c140540 2023-11-24T07:59:07.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1305700 1 --2- 172.21.15.32:0/2557134185 >> v2:172.21.15.123:3300/0 conn(0x7f439c0cf4e0 0x7f439c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:07.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 >> v1:172.21.15.32:6789/0 conn(0x7f439c140540 legacy=0x7f439c140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f4388003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388004850 0x7f4388004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4388005210 0x7f438800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438800c080 con 0x7f4388003a00 2023-11-24T07:59:07.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438800c1f0 con 0x7f4388004850 2023-11-24T07:59:07.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438800c3b0 con 0x7f4388005210 2023-11-24T07:59:07.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f4388003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1305700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388004850 0x7f4388004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a2307700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4388005210 0x7f438800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 -- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4388005210 msgr2=0x7f438800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:07.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4388005210 0x7f438800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 -- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388004850 msgr2=0x7f4388004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:07.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388004850 0x7f4388004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 -- 172.21.15.32:0/2557134185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880035b0 con 0x7f4388003a00 2023-11-24T07:59:07.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a2307700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4388005210 0x7f438800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:07.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a2307700 1 -- 172.21.15.32:0/2557134185 reap_dead start 2023-11-24T07:59:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a1b06700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f4388003e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f439800c240 tx=0x7f4388005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6188b07c23d42422 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398007c10 con 0x7f4388003a00 2023-11-24T07:59:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439800ce00 con 0x7f4388003a00 2023-11-24T07:59:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.719+0000 7f43a0b04700 1 -- 172.21.15.32:0/2557134185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398005ba0 con 0x7f4388003a00 2023-11-24T07:59:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 -- 172.21.15.32:0/2557134185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 msgr2=0x7f4388003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f4388003e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f439800c240 tx=0x7f4388005880 comp rx=0 tx=0).stop 2023-11-24T07:59:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 -- 172.21.15.32:0/2557134185 shutdown_connections 2023-11-24T07:59:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/2557134185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f4388003e20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 -- 172.21.15.32:0/2557134185 >> 172.21.15.32:0/2557134185 conn(0x7f439c072960 msgr2=0x7f439c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 -- 172.21.15.32:0/2557134185 shutdown_connections 2023-11-24T07:59:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.720+0000 7f43a3d6a700 1 -- 172.21.15.32:0/2557134185 wait complete. 2023-11-24T07:59:07.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.721+0000 7f43a3d6a700 1 Processor -- start 2023-11-24T07:59:07.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.721+0000 7f43a3d6a700 1 -- start start 2023-11-24T07:59:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a3d6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f439c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a3d6a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f439c15db30 0x7f439c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a3d6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c156360 0x7f439c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a3d6a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c1f3990 con 0x7f4388003a00 2023-11-24T07:59:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a3d6a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c15e4f0 con 0x7f439c156360 2023-11-24T07:59:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a3d6a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c0a9b30 con 0x7f439c15db30 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f439c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f439c154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57450/0 (socket says 172.21.15.32:57450) 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 -- 172.21.15.32:0/3094953782 learned_addr learned my addr 172.21.15.32:0/3094953782 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1305700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f439c15db30 0x7f439c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a2307700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c156360 0x7f439c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 -- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f439c15db30 msgr2=0x7f439c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f439c15db30 0x7f439c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 -- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c156360 msgr2=0x7f439c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c156360 0x7f439c1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1b06700 1 -- 172.21.15.32:0/3094953782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c1f4790 con 0x7f4388003a00 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.722+0000 7f43a1305700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f439c15db30 0x7f439c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.723+0000 7f43a1b06700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f439c154fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f438800c1c0 tx=0x7f4398001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.723+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398004d40 con 0x7f4388003a00 2023-11-24T07:59:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.723+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43980070d0 con 0x7f4388003a00 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.723+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398004ea0 con 0x7f4388003a00 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.723+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c1f3b80 con 0x7f4388003a00 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.723+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c1f4140 con 0x7f4388003a00 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.724+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c1f4a70 con 0x7f4388003a00 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.724+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4398007480 con 0x7f4388003a00 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.725+0000 7f43927fc700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f438006aed0 0x7f438006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.725+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f439808a550 con 0x7f4388003a00 2023-11-24T07:59:07.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.725+0000 7f43a1305700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f438006aed0 0x7f438006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.725+0000 7f43a1305700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f438006aed0 0x7f438006d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f439c15e770 tx=0x7f438c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.730+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4398055690 con 0x7f4388003a00 2023-11-24T07:59:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.929+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f439c068fb0 con 0x7f4388003a00 2023-11-24T07:59:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.929+0000 7f43927fc700 1 -- 172.21.15.32:0/3094953782 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4398058cb0 con 0x7f4388003a00 2023-11-24T07:59:07.931 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f438006aed0 msgr2=0x7f438006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f438006aed0 0x7f438006d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f439c15e770 tx=0x7f438c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 msgr2=0x7f439c154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f439c154fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f438800c1c0 tx=0x7f4398001a00 comp rx=0 tx=0).stop 2023-11-24T07:59:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 shutdown_connections 2023-11-24T07:59:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f438006aed0 0x7f438006d390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c156360 0x7f439c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4388003a00 0x7f439c154fc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 --2- 172.21.15.32:0/3094953782 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f439c15db30 0x7f439c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 >> 172.21.15.32:0/3094953782 conn(0x7f439c072960 msgr2=0x7f439c15d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 shutdown_connections 2023-11-24T07:59:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:07.931+0000 7f43a3d6a700 1 -- 172.21.15.32:0/3094953782 wait complete. 2023-11-24T07:59:08.020 INFO:tasks.cephfs_test_runner:Starting test: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T07:59:08.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.144+0000 7fc80c770700 1 Processor -- start 2023-11-24T07:59:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.144+0000 7fc80c770700 1 -- start start 2023-11-24T07:59:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.145+0000 7fc80c770700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 0x7fc8041604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.145+0000 7fc80c770700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8040b22d0 con 0x7fc80415bdd0 2023-11-24T07:59:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.145+0000 7fc80c770700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8040b25b0 con 0x7fc8040d1c70 2023-11-24T07:59:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.145+0000 7fc80c770700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040b2f50 con 0x7fc8040b3370 2023-11-24T07:59:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80a50c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc80415bdd0 0x7fc8040b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59170/0 (socket says 172.21.15.32:59170) 2023-11-24T07:59:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80a50c700 1 -- 172.21.15.32:0/1008863145 learned_addr learned my addr 172.21.15.32:0/1008863145 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc809d0b700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 0x7fc8041604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625772454 0 0) 0x7fc8040b22d0 con 0x7fc80415bdd0 2023-11-24T07:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f00035b0 con 0x7fc80415bdd0 2023-11-24T07:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 764725216 0 0) 0x7fc7f00035b0 con 0x7fc80415bdd0 2023-11-24T07:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 msgr2=0x7fc8041604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 0x7fc8041604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 >> v1:172.21.15.123:6789/0 conn(0x7fc8040d1c70 legacy=0x7fc804165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8040b2890 con 0x7fc80415bdd0 2023-11-24T07:59:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fc7f4001670 con 0x7fc80415bdd0 2023-11-24T07:59:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 >> v1:172.21.15.32:6789/0 conn(0x7fc80415bdd0 legacy=0x7fc8040b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f0003a30 0x7fc7f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc7f00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 0x7fc7f0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040b2f50 con 0x7fc7f00043e0 2023-11-24T07:59:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000ba10 con 0x7fc7f0003a30 2023-11-24T07:59:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000bbd0 con 0x7fc8040b3370 2023-11-24T07:59:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc809d0b700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc7f00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80a50c700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f0003a30 0x7fc7f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.146+0000 7fc80ad0d700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 0x7fc7f0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc809d0b700 1 -- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 msgr2=0x7fc7f0005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc809d0b700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b3370 0x7fc7f0005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc809d0b700 1 -- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f0003a30 msgr2=0x7fc7f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc809d0b700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f0003a30 0x7fc7f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc809d0b700 1 -- 172.21.15.32:0/1008863145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00035b0 con 0x7fc7f00043e0 2023-11-24T07:59:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc809d0b700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc7f00048b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc800009b60 tx=0x7fc7f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498cc3a64acfd206 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000e6c0 con 0x7fc7f00043e0 2023-11-24T07:59:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80000e820 con 0x7fc7f00043e0 2023-11-24T07:59:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc80950a700 1 -- 172.21.15.32:0/1008863145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000eb90 con 0x7fc7f00043e0 2023-11-24T07:59:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc80c770700 1 -- 172.21.15.32:0/1008863145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 msgr2=0x7fc7f00048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc80c770700 1 --2- 172.21.15.32:0/1008863145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc7f00048b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc800009b60 tx=0x7fc7f0005880 comp rx=0 tx=0).stop 2023-11-24T07:59:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.147+0000 7fc80ad0d700 1 -- 172.21.15.32:0/1008863145 reap_dead start 2023-11-24T07:59:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.148+0000 7fc80c770700 1 -- 172.21.15.32:0/1008863145 shutdown_connections 2023-11-24T07:59:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.148+0000 7fc80c770700 1 -- 172.21.15.32:0/1008863145 >> 172.21.15.32:0/1008863145 conn(0x7fc8040731a0 msgr2=0x7fc80409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.148+0000 7fc80c770700 1 -- 172.21.15.32:0/1008863145 shutdown_connections 2023-11-24T07:59:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.148+0000 7fc80c770700 1 -- 172.21.15.32:0/1008863145 wait complete. 2023-11-24T07:59:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.149+0000 7fc80c770700 1 Processor -- start 2023-11-24T07:59:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.149+0000 7fc80c770700 1 -- start start 2023-11-24T07:59:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc8040cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b48d0 0x7fc80415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8040b4d50 0x7fc80415f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80415daf0 con 0x7fc7f00043e0 2023-11-24T07:59:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040ce010 con 0x7fc8040b4d50 2023-11-24T07:59:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040cd400 con 0x7fc8040b48d0 2023-11-24T07:59:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc8040cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc8040cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57470/0 (socket says 172.21.15.32:57470) 2023-11-24T07:59:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 -- 172.21.15.32:0/2565786826 learned_addr learned my addr 172.21.15.32:0/2565786826 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80ad0d700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8040b4d50 0x7fc80415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc809d0b700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b48d0 0x7fc80415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 -- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b48d0 msgr2=0x7fc80415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b48d0 0x7fc80415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 -- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8040b4d50 msgr2=0x7fc80415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8040b4d50 0x7fc80415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 -- 172.21.15.32:0/2565786826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8040cd6e0 con 0x7fc7f00043e0 2023-11-24T07:59:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc809d0b700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b48d0 0x7fc80415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80a50c700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc8040cabb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4007290 tx=0x7fc7f4007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400f8d0 con 0x7fc7f00043e0 2023-11-24T07:59:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f400fa50 con 0x7fc7f00043e0 2023-11-24T07:59:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400fe00 con 0x7fc7f00043e0 2023-11-24T07:59:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8040cd9c0 con 0x7fc7f00043e0 2023-11-24T07:59:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.150+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8040cd090 con 0x7fc7f00043e0 2023-11-24T07:59:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.151+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc804067f60 con 0x7fc7f00043e0 2023-11-24T07:59:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.152+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc7f401d460 con 0x7fc7f00043e0 2023-11-24T07:59:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.152+0000 7fc7faffd700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc7e806aed0 0x7fc7e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.152+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7f40895a0 con 0x7fc7f00043e0 2023-11-24T07:59:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.152+0000 7fc809d0b700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc7e806aed0 0x7fc7e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.153+0000 7fc809d0b700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc7e806aed0 0x7fc7e806d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc800000c00 tx=0x7fc800008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.156+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7f4054650 con 0x7fc7f00043e0 2023-11-24T07:59:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.356+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc80415ae10 con 0x7fc7f00043e0 2023-11-24T07:59:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.357+0000 7fc7faffd700 1 -- 172.21.15.32:0/2565786826 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fc7f4057c70 con 0x7fc7f00043e0 2023-11-24T07:59:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc7e806aed0 msgr2=0x7fc7e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc7e806aed0 0x7fc7e806d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc800000c00 tx=0x7fc800008030 comp rx=0 tx=0).stop 2023-11-24T07:59:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 msgr2=0x7fc8040cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc8040cabb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4007290 tx=0x7fc7f4007650 comp rx=0 tx=0).stop 2023-11-24T07:59:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 shutdown_connections 2023-11-24T07:59:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fc7e806aed0 0x7fc7e806d390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8040b4d50 0x7fc80415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7f00043e0 0x7fc8040cabb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 --2- 172.21.15.32:0/2565786826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc8040b48d0 0x7fc80415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 >> 172.21.15.32:0/2565786826 conn(0x7fc8040731a0 msgr2=0x7fc8040b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 shutdown_connections 2023-11-24T07:59:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.359+0000 7fc80c770700 1 -- 172.21.15.32:0/2565786826 wait complete. 2023-11-24T07:59:08.375 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:08.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.504+0000 7fbf349bd700 1 Processor -- start 2023-11-24T07:59:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.504+0000 7fbf349bd700 1 -- start start 2023-11-24T07:59:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf349bd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 0x7fbf3015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf349bd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbf300b3c60 0x7fbf300b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf349bd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300ae5e0 con 0x7fbf301587f0 2023-11-24T07:59:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf349bd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf300b4610 con 0x7fbf300b3c60 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf349bd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3015be30 con 0x7fbf3015b370 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2dd9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 0x7fbf3015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2ed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbf300b3c60 0x7fbf300b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2dd9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 0x7fbf3015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:52732/0 (socket says 172.21.15.32:52732) 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2e59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbf301587f0 0x7fbf3006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:45944/0 (socket says 172.21.15.32:45944) 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2dd9b700 1 -- 172.21.15.32:0/4168464469 learned_addr learned my addr 172.21.15.32:0/4168464469 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2ed9d700 1 -- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 msgr2=0x7fbf3015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2ed9d700 1 --2- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 0x7fbf3015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2ed9d700 1 -- 172.21.15.32:0/4168464469 >> v1:172.21.15.123:6789/0 conn(0x7fbf301587f0 legacy=0x7fbf3006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf2ed9d700 1 -- 172.21.15.32:0/4168464469 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3015c110 con 0x7fbf300b3c60 2023-11-24T07:59:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.505+0000 7fbf2d59a700 1 -- 172.21.15.32:0/4168464469 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740502412 0 0) 0x7fbf300ae5e0 con 0x7fbf301587f0 2023-11-24T07:59:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf2dd9b700 1 --2- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 0x7fbf3015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf2ed9d700 1 --2- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf300b3c60 0x7fbf300b40d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009d90 tx=0x7fbf2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d4500218504f6ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf2d59a700 1 -- 172.21.15.32:0/4168464469 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800e070 con 0x7fbf300b3c60 2023-11-24T07:59:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf2d59a700 1 -- 172.21.15.32:0/4168464469 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2800bb20 con 0x7fbf300b3c60 2023-11-24T07:59:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf2d59a700 1 -- 172.21.15.32:0/4168464469 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28012430 con 0x7fbf300b3c60 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf349bd700 1 -- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf300b3c60 msgr2=0x7fbf300b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf349bd700 1 --2- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf300b3c60 0x7fbf300b40d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009d90 tx=0x7fbf2800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf349bd700 1 -- 172.21.15.32:0/4168464469 shutdown_connections 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf349bd700 1 --2- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf300b3c60 0x7fbf300b40d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.506+0000 7fbf349bd700 1 --2- 172.21.15.32:0/4168464469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf3015b370 0x7fbf3015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.507+0000 7fbf349bd700 1 -- 172.21.15.32:0/4168464469 >> 172.21.15.32:0/4168464469 conn(0x7fbf300d0bf0 msgr2=0x7fbf300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.507+0000 7fbf349bd700 1 -- 172.21.15.32:0/4168464469 shutdown_connections 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.507+0000 7fbf349bd700 1 -- 172.21.15.32:0/4168464469 wait complete. 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.508+0000 7fbf349bd700 1 Processor -- start 2023-11-24T07:59:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.508+0000 7fbf349bd700 1 -- start start 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf349bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 0x7fbf301f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf349bd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf301587f0 0x7fbf3006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf349bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf3015b370 0x7fbf301f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf349bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30145e90 con 0x7fbf300b3c60 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf349bd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301460a0 con 0x7fbf3015b370 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf349bd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30146380 con 0x7fbf301587f0 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 0x7fbf301f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 0x7fbf301f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57484/0 (socket says 172.21.15.32:57484) 2023-11-24T07:59:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 -- 172.21.15.32:0/193336312 learned_addr learned my addr 172.21.15.32:0/193336312 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2dd9b700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf301587f0 0x7fbf3006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2ed9d700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf3015b370 0x7fbf301f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 -- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf301587f0 msgr2=0x7fbf3006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf301587f0 0x7fbf3006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 -- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf3015b370 msgr2=0x7fbf301f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf3015b370 0x7fbf301f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 -- 172.21.15.32:0/193336312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009550 con 0x7fbf300b3c60 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2ed9d700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf3015b370 0x7fbf301f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.509+0000 7fbf2e59c700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 0x7fbf301f3da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf200011e0 tx=0x7fbf2000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.510+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20004c80 con 0x7fbf300b3c60 2023-11-24T07:59:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.510+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20004e00 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.510+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf28009a70 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.510+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20007310 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.510+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf30146940 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.511+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf30068fb0 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.511+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf20007470 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.511+0000 7fbf1effd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf1006aed0 0x7fbf1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.511+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf20089380 con 0x7fbf300b3c60 2023-11-24T07:59:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.512+0000 7fbf2dd9b700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf1006aed0 0x7fbf1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.512+0000 7fbf2dd9b700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf1006aed0 0x7fbf1006d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf30068220 tx=0x7fbf24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.516+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf20054430 con 0x7fbf300b3c60 2023-11-24T07:59:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.714+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf3006cb30 con 0x7fbf300b3c60 2023-11-24T07:59:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.715+0000 7fbf1effd700 1 -- 172.21.15.32:0/193336312 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf20057a50 con 0x7fbf300b3c60 2023-11-24T07:59:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:08.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf1006aed0 msgr2=0x7fbf1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf1006aed0 0x7fbf1006d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf30068220 tx=0x7fbf24008040 comp rx=0 tx=0).stop 2023-11-24T07:59:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 msgr2=0x7fbf301f3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 0x7fbf301f3da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf200011e0 tx=0x7fbf2000cae0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 shutdown_connections 2023-11-24T07:59:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf1006aed0 0x7fbf1006d390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf3015b370 0x7fbf301f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf300b3c60 0x7fbf301f3da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 --2- 172.21.15.32:0/193336312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf301587f0 0x7fbf3006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 >> 172.21.15.32:0/193336312 conn(0x7fbf300d0bf0 msgr2=0x7fbf3006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 shutdown_connections 2023-11-24T07:59:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.717+0000 7fbf349bd700 1 -- 172.21.15.32:0/193336312 wait complete. 2023-11-24T07:59:08.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.857+0000 7f4b2f113700 1 Processor -- start 2023-11-24T07:59:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.857+0000 7f4b2f113700 1 -- start start 2023-11-24T07:59:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b2f113700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b280cf610 0x7f4b280add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b2f113700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2806b080 con 0x7f4b2815f9e0 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b2f113700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2815c290 con 0x7f4b28073f90 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b2f113700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b280cf9f0 con 0x7f4b280cf610 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b27fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4b2815f9e0 0x7f4b280b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59204/0 (socket says 172.21.15.32:59204) 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b27fff700 1 -- 172.21.15.32:0/579866995 learned_addr learned my addr 172.21.15.32:0/579866995 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b2ceaf700 1 --2- 172.21.15.32:0/579866995 >> v2:172.21.15.123:3300/0 conn(0x7f4b280cf610 0x7f4b280add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246758990 0 0) 0x7f4b2806b080 con 0x7f4b2815f9e0 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b180035b0 con 0x7f4b2815f9e0 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4190926775 0 0) 0x7f4b180035b0 con 0x7f4b2815f9e0 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 >> v2:172.21.15.123:3300/0 conn(0x7f4b280cf610 msgr2=0x7f4b280add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 --2- 172.21.15.32:0/579866995 >> v2:172.21.15.123:3300/0 conn(0x7f4b280cf610 0x7f4b280add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 >> v1:172.21.15.123:6789/0 conn(0x7f4b28073f90 legacy=0x7f4b2806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2815ff80 con 0x7f4b2815f9e0 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f4b14001670 con 0x7f4b2815f9e0 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 >> v1:172.21.15.32:6789/0 conn(0x7f4b2815f9e0 legacy=0x7f4b280b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b18003a00 0x7f4b18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18004850 0x7f4b18004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b18005210 0x7f4b1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280cf9f0 con 0x7f4b18005210 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1800c0b0 con 0x7f4b18003a00 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1800c270 con 0x7f4b18004850 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.858+0000 7f4b2ceaf700 1 --2- 172.21.15.32:0/579866995 >> v2:172.21.15.123:3300/0 conn(0x7f4b280cf610 0x7f4b280add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b18005210 0x7f4b1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b27fff700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18004850 0x7f4b18004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2ceaf700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b18003a00 0x7f4b18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 -- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18004850 msgr2=0x7f4b18004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18004850 0x7f4b18004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 -- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b18003a00 msgr2=0x7f4b18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b18003a00 0x7f4b18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 -- 172.21.15.32:0/579866995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b180035b0 con 0x7f4b18005210 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 -- 172.21.15.32:0/579866995 reap_dead start 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b2d6b0700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b18005210 0x7f4b1800ba10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000c190 tx=0x7f4b18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fe6f316e74cdcc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b20001db0 con 0x7f4b18005210 2023-11-24T07:59:08.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2000ccf0 con 0x7f4b18005210 2023-11-24T07:59:08.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.859+0000 7f4b277fe700 1 -- 172.21.15.32:0/579866995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003c40 con 0x7f4b18005210 2023-11-24T07:59:08.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 -- 172.21.15.32:0/579866995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b18005210 msgr2=0x7f4b1800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b18005210 0x7f4b1800ba10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000c190 tx=0x7f4b18005880 comp rx=0 tx=0).stop 2023-11-24T07:59:08.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 -- 172.21.15.32:0/579866995 shutdown_connections 2023-11-24T07:59:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 --2- 172.21.15.32:0/579866995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b18005210 0x7f4b1800ba10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 -- 172.21.15.32:0/579866995 >> 172.21.15.32:0/579866995 conn(0x7f4b280d0bf0 msgr2=0x7f4b280d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 -- 172.21.15.32:0/579866995 shutdown_connections 2023-11-24T07:59:08.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.860+0000 7f4b2f113700 1 -- 172.21.15.32:0/579866995 wait complete. 2023-11-24T07:59:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.861+0000 7f4b2f113700 1 Processor -- start 2023-11-24T07:59:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.861+0000 7f4b2f113700 1 -- start start 2023-11-24T07:59:08.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.862+0000 7f4b2f113700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18005210 0x7f4b280b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 0x7f4b28154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b281f5460 0x7f4b281f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280b2790 con 0x7f4b280affa0 2023-11-24T07:59:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b281f1e10 con 0x7f4b281f5460 2023-11-24T07:59:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280b16c0 con 0x7f4b18005210 2023-11-24T07:59:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 0x7f4b28154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 0x7f4b28154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57506/0 (socket says 172.21.15.32:57506) 2023-11-24T07:59:08.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 -- 172.21.15.32:0/1689624732 learned_addr learned my addr 172.21.15.32:0/1689624732 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2d6b0700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b281f5460 0x7f4b281f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2ceaf700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18005210 0x7f4b280b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 -- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18005210 msgr2=0x7f4b280b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18005210 0x7f4b280b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 -- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b281f5460 msgr2=0x7f4b281f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b281f5460 0x7f4b281f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 -- 172.21.15.32:0/1689624732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b280b19a0 con 0x7f4b280affa0 2023-11-24T07:59:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2d6b0700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b281f5460 0x7f4b281f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b27fff700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 0x7f4b28154870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b14007290 tx=0x7f4b140075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011580 con 0x7f4b280affa0 2023-11-24T07:59:08.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b140116e0 con 0x7f4b280affa0 2023-11-24T07:59:08.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b280b1c80 con 0x7f4b280affa0 2023-11-24T07:59:08.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011a70 con 0x7f4b280affa0 2023-11-24T07:59:08.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.863+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b28152300 con 0x7f4b280affa0 2023-11-24T07:59:08.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.864+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b28152490 con 0x7f4b280affa0 2023-11-24T07:59:08.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.865+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b14011bd0 con 0x7f4b280affa0 2023-11-24T07:59:08.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.865+0000 7f4b257fa700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4b0c06aed0 0x7f4b0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.865+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b140893f0 con 0x7f4b280affa0 2023-11-24T07:59:08.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.865+0000 7f4b2ceaf700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4b0c06aed0 0x7f4b0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.866+0000 7f4b2ceaf700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4b0c06aed0 0x7f4b0c06d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009470 tx=0x7f4b1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:08.870+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b14054530 con 0x7f4b280affa0 2023-11-24T07:59:09.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.073+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b28068fb0 con 0x7f4b280affa0 2023-11-24T07:59:09.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.073+0000 7f4b257fa700 1 -- 172.21.15.32:0/1689624732 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b14057b50 con 0x7f4b280affa0 2023-11-24T07:59:09.075 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4b0c06aed0 msgr2=0x7f4b0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4b0c06aed0 0x7f4b0c06d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009470 tx=0x7f4b1c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 msgr2=0x7f4b28154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 0x7f4b28154870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b14007290 tx=0x7f4b140075a0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 shutdown_connections 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4b0c06aed0 0x7f4b0c06d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b281f5460 0x7f4b281f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b18005210 0x7f4b280b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 --2- 172.21.15.32:0/1689624732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b280affa0 0x7f4b28154870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 >> 172.21.15.32:0/1689624732 conn(0x7f4b280d0bf0 msgr2=0x7f4b280b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 shutdown_connections 2023-11-24T07:59:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.075+0000 7f4b2f113700 1 -- 172.21.15.32:0/1689624732 wait complete. 2023-11-24T07:59:09.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.215+0000 7f7e7b96b700 1 Processor -- start 2023-11-24T07:59:09.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.215+0000 7f7e7b96b700 1 -- start start 2023-11-24T07:59:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e7b96b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e7415e160 0x7f7e740b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e7b96b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e740746a0 0x7f7e7406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e7b96b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e740b3c60 con 0x7f7e740afc90 2023-11-24T07:59:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e7b96b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e740b4500 con 0x7f7e740746a0 2023-11-24T07:59:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e7b96b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740b58c0 con 0x7f7e7415e160 2023-11-24T07:59:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e740746a0 0x7f7e7406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e740746a0 0x7f7e7406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57514/0 (socket says 172.21.15.32:57514) 2023-11-24T07:59:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 -- 172.21.15.32:0/1803520604 learned_addr learned my addr 172.21.15.32:0/1803520604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:09.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79707700 1 --2- 172.21.15.32:0/1803520604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e7415e160 0x7f7e740b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 -- 172.21.15.32:0/1803520604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e7415e160 msgr2=0x7f7e740b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:09.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 --2- 172.21.15.32:0/1803520604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e7415e160 0x7f7e740b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 -- 172.21.15.32:0/1803520604 >> v1:172.21.15.32:6789/0 conn(0x7f7e740afc90 legacy=0x7f7e74073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.216+0000 7f7e79f08700 1 -- 172.21.15.32:0/1803520604 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7406bdc0 con 0x7f7e740746a0 2023-11-24T07:59:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e79f08700 1 --2- 172.21.15.32:0/1803520604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e740746a0 0x7f7e7406d930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f9f7a8ee006e71 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e6bfff700 1 -- 172.21.15.32:0/1803520604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00e070 con 0x7f7e740746a0 2023-11-24T07:59:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e6bfff700 1 -- 172.21.15.32:0/1803520604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c00b910 con 0x7f7e740746a0 2023-11-24T07:59:09.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e6bfff700 1 -- 172.21.15.32:0/1803520604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00bcc0 con 0x7f7e740746a0 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 -- 172.21.15.32:0/1803520604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e740746a0 msgr2=0x7f7e7406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/1803520604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e740746a0 0x7f7e7406d930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 -- 172.21.15.32:0/1803520604 shutdown_connections 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/1803520604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e740746a0 0x7f7e7406d930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/1803520604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e7415e160 0x7f7e740b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 -- 172.21.15.32:0/1803520604 >> 172.21.15.32:0/1803520604 conn(0x7f7e740d0bf0 msgr2=0x7f7e740d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 -- 172.21.15.32:0/1803520604 shutdown_connections 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.217+0000 7f7e7b96b700 1 -- 172.21.15.32:0/1803520604 wait complete. 2023-11-24T07:59:09.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.219+0000 7f7e7b96b700 1 Processor -- start 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.219+0000 7f7e7b96b700 1 -- start start 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e7b96b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e740746a0 0x7f7e74158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e7b96b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e740afc90 0x7f7e74160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e7b96b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 0x7f7e74160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e7b96b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74160200 con 0x7f7e7415e160 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e7b96b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7415f5f0 con 0x7f7e740746a0 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e7b96b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7415f8d0 con 0x7f7e740afc90 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 0x7f7e74160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 0x7f7e74160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57520/0 (socket says 172.21.15.32:57520) 2023-11-24T07:59:09.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 -- 172.21.15.32:0/3204746588 learned_addr learned my addr 172.21.15.32:0/3204746588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79707700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e740746a0 0x7f7e74158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e78f06700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e740afc90 0x7f7e74160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 -- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e740afc90 msgr2=0x7f7e74160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e740afc90 0x7f7e74160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 -- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e740746a0 msgr2=0x7f7e74158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e740746a0 0x7f7e74158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79f08700 1 -- 172.21.15.32:0/3204746588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c009b30 con 0x7f7e7415e160 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.220+0000 7f7e79707700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e740746a0 0x7f7e74158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.221+0000 7f7e79f08700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 0x7f7e74160fa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c012000 tx=0x7f7e6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.221+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c004920 con 0x7f7e7415e160 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.221+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c0099b0 con 0x7f7e7415e160 2023-11-24T07:59:09.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.221+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c01c680 con 0x7f7e7415e160 2023-11-24T07:59:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.221+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7415fbb0 con 0x7f7e7415e160 2023-11-24T07:59:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.221+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e74158dd0 con 0x7f7e7415e160 2023-11-24T07:59:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.222+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e74074e60 con 0x7f7e7415e160 2023-11-24T07:59:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.222+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e6c01c7e0 con 0x7f7e7415e160 2023-11-24T07:59:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.223+0000 7f7e69ffb700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7e5806aed0 0x7f7e5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.223+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e6c08b370 con 0x7f7e7415e160 2023-11-24T07:59:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.223+0000 7f7e79707700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7e5806aed0 0x7f7e5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.224+0000 7f7e79707700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7e5806aed0 0x7f7e5806d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009ac0 tx=0x7f7e64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.227+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e6c056410 con 0x7f7e7415e160 2023-11-24T07:59:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.431+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e74068fb0 con 0x7f7e7415e160 2023-11-24T07:59:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.431+0000 7f7e69ffb700 1 -- 172.21.15.32:0/3204746588 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e6c059a30 con 0x7f7e7415e160 2023-11-24T07:59:09.433 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7e5806aed0 msgr2=0x7f7e5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7e5806aed0 0x7f7e5806d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009ac0 tx=0x7f7e64008040 comp rx=0 tx=0).stop 2023-11-24T07:59:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 msgr2=0x7f7e74160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 0x7f7e74160fa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c012000 tx=0x7f7e6c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 shutdown_connections 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7e5806aed0 0x7f7e5806d390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e7415e160 0x7f7e74160fa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e740746a0 0x7f7e74158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 --2- 172.21.15.32:0/3204746588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e740afc90 0x7f7e74160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.433+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 >> 172.21.15.32:0/3204746588 conn(0x7f7e740d0bf0 msgr2=0x7f7e740d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.434+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 shutdown_connections 2023-11-24T07:59:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.434+0000 7f7e7b96b700 1 -- 172.21.15.32:0/3204746588 wait complete. 2023-11-24T07:59:09.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2023-11-24T07:59:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.570+0000 7fadb8e36700 1 Processor -- start 2023-11-24T07:59:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.571+0000 7fadb8e36700 1 -- start start 2023-11-24T07:59:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.571+0000 7fadb8e36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadb415e2a0 0x7fadb406ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb8e36700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb406e7f0 con 0x7fadb4073f90 2023-11-24T07:59:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb8e36700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb406dec0 con 0x7fadb40ce090 2023-11-24T07:59:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb8e36700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb406e1a0 con 0x7fadb415e2a0 2023-11-24T07:59:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadb415e2a0 0x7fadb406ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadb415e2a0 0x7fadb406ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57524/0 (socket says 172.21.15.32:57524) 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 -- 172.21.15.32:0/2245563455 learned_addr learned my addr 172.21.15.32:0/2245563455 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fadb4073f90 0x7fadb40cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59240/0 (socket says 172.21.15.32:59240) 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 -- 172.21.15.32:0/2245563455 >> v1:172.21.15.123:6789/0 conn(0x7fadb40ce090 legacy=0x7fadb406ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 -- 172.21.15.32:0/2245563455 >> v1:172.21.15.32:6789/0 conn(0x7fadb4073f90 legacy=0x7fadb40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 -- 172.21.15.32:0/2245563455 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb406e480 con 0x7fadb415e2a0 2023-11-24T07:59:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb259c700 1 --2- 172.21.15.32:0/2245563455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb415e2a0 0x7fadb406ba70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fada80098f0 tx=0x7fada8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71212218a842157 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb159a700 1 -- 172.21.15.32:0/2245563455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800b8a0 con 0x7fadb415e2a0 2023-11-24T07:59:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb159a700 1 -- 172.21.15.32:0/2245563455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada800ba00 con 0x7fadb415e2a0 2023-11-24T07:59:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.572+0000 7fadb159a700 1 -- 172.21.15.32:0/2245563455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800bd90 con 0x7fadb415e2a0 2023-11-24T07:59:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 -- 172.21.15.32:0/2245563455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb415e2a0 msgr2=0x7fadb406ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 --2- 172.21.15.32:0/2245563455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb415e2a0 0x7fadb406ba70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fada80098f0 tx=0x7fada8009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 -- 172.21.15.32:0/2245563455 shutdown_connections 2023-11-24T07:59:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 --2- 172.21.15.32:0/2245563455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb415e2a0 0x7fadb406ba70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 -- 172.21.15.32:0/2245563455 >> 172.21.15.32:0/2245563455 conn(0x7fadb40cd570 msgr2=0x7fadb40cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 -- 172.21.15.32:0/2245563455 shutdown_connections 2023-11-24T07:59:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.573+0000 7fadb8e36700 1 -- 172.21.15.32:0/2245563455 wait complete. 2023-11-24T07:59:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.574+0000 7fadb8e36700 1 Processor -- start 2023-11-24T07:59:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.574+0000 7fadb8e36700 1 -- start start 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb8e36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 0x7fadb4145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb8e36700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb40ce090 0x7fadb4157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb8e36700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadb415e2a0 0x7fadb4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb8e36700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40ad870 con 0x7fadb4073f90 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb8e36700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4161620 con 0x7fadb40ce090 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb8e36700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb40ada80 con 0x7fadb415e2a0 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 0x7fadb4145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 0x7fadb4145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57530/0 (socket says 172.21.15.32:57530) 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb259c700 1 -- 172.21.15.32:0/3035374415 learned_addr learned my addr 172.21.15.32:0/3035374415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb2d9d700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadb415e2a0 0x7fadb4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.575+0000 7fadb1d9b700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb40ce090 0x7fadb4157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb259c700 1 -- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadb415e2a0 msgr2=0x7fadb4157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb259c700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadb415e2a0 0x7fadb4157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb259c700 1 -- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb40ce090 msgr2=0x7fadb4157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb259c700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb40ce090 0x7fadb4157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb259c700 1 -- 172.21.15.32:0/3035374415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada8009550 con 0x7fadb4073f90 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb259c700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 0x7fadb4145ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fada8009130 tx=0x7fada8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80036a0 con 0x7fadb4073f90 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada801e070 con 0x7fadb4073f90 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb40add60 con 0x7fadb4073f90 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8003a40 con 0x7fadb4073f90 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.576+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb4155b30 con 0x7fadb4073f90 2023-11-24T07:59:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.577+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb40ae3b0 con 0x7fadb4073f90 2023-11-24T07:59:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.577+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fada8003ba0 con 0x7fadb4073f90 2023-11-24T07:59:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.578+0000 7fada2ffd700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fad9406aed0 0x7fad9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.578+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fada8020070 con 0x7fadb4073f90 2023-11-24T07:59:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.578+0000 7fadb1d9b700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fad9406aed0 0x7fad9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.579+0000 7fadb1d9b700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fad9406aed0 0x7fad9406d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fadb41558d0 tx=0x7fada4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.583+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fada8055bd0 con 0x7fadb4073f90 2023-11-24T07:59:09.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:09.873+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7fadb4067f60 con 0x7fadb4073f90 2023-11-24T07:59:10.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.594+0000 7fada2ffd700 1 -- 172.21.15.32:0/3035374415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7fada8015020 con 0x7fadb4073f90 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fad9406aed0 msgr2=0x7fad9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fad9406aed0 0x7fad9406d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fadb41558d0 tx=0x7fada4009040 comp rx=0 tx=0).stop 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 msgr2=0x7fadb4145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 0x7fadb4145ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fada8009130 tx=0x7fada8004750 comp rx=0 tx=0).stop 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 shutdown_connections 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fad9406aed0 0x7fad9406d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadb415e2a0 0x7fadb4157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadb4073f90 0x7fadb4145ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 --2- 172.21.15.32:0/3035374415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb40ce090 0x7fadb4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 >> 172.21.15.32:0/3035374415 conn(0x7fadb40cd570 msgr2=0x7fadb40cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 shutdown_connections 2023-11-24T07:59:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.597+0000 7fadb8e36700 1 -- 172.21.15.32:0/3035374415 wait complete. 2023-11-24T07:59:10.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:59:10.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.749+0000 7fb1498b7700 1 Processor -- start 2023-11-24T07:59:10.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.749+0000 7fb1498b7700 1 -- start start 2023-11-24T07:59:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1498b7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb14415e140 0x7fb1440d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1498b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1440d39d0 0x7fb1440d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1498b7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1440b80e0 con 0x7fb1440b98c0 2023-11-24T07:59:10.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1498b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14415e520 con 0x7fb1440d39d0 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1498b7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1440d2930 con 0x7fb14415e140 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1440d39d0 0x7fb1440d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1440d39d0 0x7fb1440d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57540/0 (socket says 172.21.15.32:57540) 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 -- 172.21.15.32:0/2015747019 learned_addr learned my addr 172.21.15.32:0/2015747019 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb142ffd700 1 --2- 172.21.15.32:0/2015747019 >> v2:172.21.15.123:3300/0 conn(0x7fb14415e140 0x7fb1440d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 -- 172.21.15.32:0/2015747019 >> v2:172.21.15.123:3300/0 conn(0x7fb14415e140 msgr2=0x7fb1440d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 --2- 172.21.15.32:0/2015747019 >> v2:172.21.15.123:3300/0 conn(0x7fb14415e140 0x7fb1440d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 -- 172.21.15.32:0/2015747019 >> v1:172.21.15.123:6789/0 conn(0x7fb1440b98c0 legacy=0x7fb1440b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 -- 172.21.15.32:0/2015747019 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440d2c10 con 0x7fb1440d39d0 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb142ffd700 1 --2- 172.21.15.32:0/2015747019 >> v2:172.21.15.123:3300/0 conn(0x7fb14415e140 0x7fb1440d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.750+0000 7fb1427fc700 1 --2- 172.21.15.32:0/2015747019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440d39d0 0x7fb1440d4400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb1440d5030 tx=0x7fb134009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cd5ed110e52109 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb141ffb700 1 -- 172.21.15.32:0/2015747019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400b870 con 0x7fb1440d39d0 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb141ffb700 1 -- 172.21.15.32:0/2015747019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13400b9d0 con 0x7fb1440d39d0 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb141ffb700 1 -- 172.21.15.32:0/2015747019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400bd60 con 0x7fb1440d39d0 2023-11-24T07:59:10.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 -- 172.21.15.32:0/2015747019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440d39d0 msgr2=0x7fb1440d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 --2- 172.21.15.32:0/2015747019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440d39d0 0x7fb1440d4400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb1440d5030 tx=0x7fb134009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 -- 172.21.15.32:0/2015747019 shutdown_connections 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 --2- 172.21.15.32:0/2015747019 >> v2:172.21.15.123:3300/0 conn(0x7fb14415e140 0x7fb1440d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 --2- 172.21.15.32:0/2015747019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440d39d0 0x7fb1440d4400 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 -- 172.21.15.32:0/2015747019 >> 172.21.15.32:0/2015747019 conn(0x7fb14406b410 msgr2=0x7fb14406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 -- 172.21.15.32:0/2015747019 shutdown_connections 2023-11-24T07:59:10.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.751+0000 7fb1498b7700 1 -- 172.21.15.32:0/2015747019 wait complete. 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.752+0000 7fb1498b7700 1 Processor -- start 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 -- start start 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 0x7fb1440b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1440d39d0 0x7fb14415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb14415e140 0x7fb1440b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440ce050 con 0x7fb1440b98c0 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440cdbc0 con 0x7fb1440d39d0 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1498b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440ccfb0 con 0x7fb14415e140 2023-11-24T07:59:10.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb142ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 0x7fb1440b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb142ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 0x7fb1440b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57544/0 (socket says 172.21.15.32:57544) 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb142ffd700 1 -- 172.21.15.32:0/1229252956 learned_addr learned my addr 172.21.15.32:0/1229252956 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1427fc700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1440d39d0 0x7fb14415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.753+0000 7fb1437fe700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb14415e140 0x7fb1440b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb142ffd700 1 -- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb14415e140 msgr2=0x7fb1440b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb142ffd700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb14415e140 0x7fb1440b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb142ffd700 1 -- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1440d39d0 msgr2=0x7fb14415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb142ffd700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1440d39d0 0x7fb14415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb142ffd700 1 -- 172.21.15.32:0/1229252956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134009550 con 0x7fb1440b98c0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb142ffd700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 0x7fb1440b6510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e810 tx=0x7fb13800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800c020 con 0x7fb1440b98c0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb138004250 con 0x7fb1440b98c0 2023-11-24T07:59:10.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1440cd290 con 0x7fb1440b98c0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1380036a0 con 0x7fb1440b98c0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.754+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1440cd850 con 0x7fb1440b98c0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.755+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144068fb0 con 0x7fb1440b98c0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.756+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb138003800 con 0x7fb1440b98c0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.756+0000 7fb12bfff700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb13006aed0 0x7fb13006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.756+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb138089170 con 0x7fb1440b98c0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.756+0000 7fb1427fc700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb13006aed0 0x7fb13006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.757+0000 7fb1427fc700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb13006aed0 0x7fb13006d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb134000c00 tx=0x7fb134008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.760+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1380542b0 con 0x7fb1440b98c0 2023-11-24T07:59:10.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.963+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb144161480 con 0x7fb1440b98c0 2023-11-24T07:59:10.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.966+0000 7fb12bfff700 1 -- 172.21.15.32:0/1229252956 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1380578d0 con 0x7fb1440b98c0 2023-11-24T07:59:10.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb13006aed0 msgr2=0x7fb13006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb13006aed0 0x7fb13006d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb134000c00 tx=0x7fb134008030 comp rx=0 tx=0).stop 2023-11-24T07:59:10.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 msgr2=0x7fb1440b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 0x7fb1440b6510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e810 tx=0x7fb13800eb20 comp rx=0 tx=0).stop 2023-11-24T07:59:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 shutdown_connections 2023-11-24T07:59:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb13006aed0 0x7fb13006d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb14415e140 0x7fb1440b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1440b98c0 0x7fb1440b6510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 --2- 172.21.15.32:0/1229252956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1440d39d0 0x7fb14415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 >> 172.21.15.32:0/1229252956 conn(0x7fb14406b410 msgr2=0x7fb1440ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 shutdown_connections 2023-11-24T07:59:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:10.968+0000 7fb1498b7700 1 -- 172.21.15.32:0/1229252956 wait complete. 2023-11-24T07:59:10.982 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:59:10.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:11.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:11.076 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:59:11.077 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:11.077 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:11.095 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:11.096 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:11.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:11.138 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:11.153 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:11.153 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:11.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.409+0000 7f69027ce700 1 Processor -- start 2023-11-24T07:59:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.409+0000 7f69027ce700 1 -- start start 2023-11-24T07:59:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.409+0000 7f69027ce700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68fc15e2a0 0x7f68fc06ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.409+0000 7f69027ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68fc074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f69027ce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc06e7f0 con 0x7f68fc0ce090 2023-11-24T07:59:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f69027ce700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc074960 con 0x7f68fc15e2a0 2023-11-24T07:59:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f69027ce700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc06dec0 con 0x7f68fc073f90 2023-11-24T07:59:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68fb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68fc0ce090 0x7f68fc06ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59264/0 (socket says 172.21.15.32:59264) 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68fb7fe700 1 -- 172.21.15.32:0/1212485202 learned_addr learned my addr 172.21.15.32:0/1212485202 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102118988 0 0) 0x7f68fc06e7f0 con 0x7f68fc0ce090 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68fbfff700 1 --2- 172.21.15.32:0/1212485202 >> v2:172.21.15.123:3300/0 conn(0x7f68fc15e2a0 0x7f68fc06ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e40035b0 con 0x7f68fc0ce090 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f6900d6b700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68fc074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169521226 0 0) 0x7f68e40035b0 con 0x7f68fc0ce090 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 msgr2=0x7f68fc074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68fc074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 >> v2:172.21.15.123:3300/0 conn(0x7f68fc15e2a0 msgr2=0x7f68fc06ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 --2- 172.21.15.32:0/1212485202 >> v2:172.21.15.123:3300/0 conn(0x7f68fc15e2a0 0x7f68fc06ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc06e1a0 con 0x7f68fc0ce090 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f68f0001670 con 0x7f68fc0ce090 2023-11-24T07:59:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 >> v1:172.21.15.32:6789/0 conn(0x7f68fc0ce090 legacy=0x7f68fc06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68e4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e4003f80 0x7f68e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4004d90 0x7f68e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc074960 con 0x7f68e4004d90 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc06dec0 con 0x7f68e4003f80 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e400ba60 con 0x7f68fc073f90 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.410+0000 7f6900d6b700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4004d90 0x7f68e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f68fbfff700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68e4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f68fb7fe700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e4003f80 0x7f68e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f6900d6b700 1 -- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 msgr2=0x7f68e4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f6900d6b700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68e4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f6900d6b700 1 -- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e4003f80 msgr2=0x7f68e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f6900d6b700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e4003f80 0x7f68e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f6900d6b700 1 -- 172.21.15.32:0/1212485202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40035b0 con 0x7f68e4004d90 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f68fbfff700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc073f90 0x7f68e4003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f6900d6b700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4004d90 0x7f68e4005210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68f400e170 tx=0x7f68e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de416ee1bc7a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f400ec90 con 0x7f68e4004d90 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f400f040 con 0x7f68e4004d90 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.411+0000 7f68faffd700 1 -- 172.21.15.32:0/1212485202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4014480 con 0x7f68e4004d90 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f69027ce700 1 -- 172.21.15.32:0/1212485202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4004d90 msgr2=0x7f68e4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f69027ce700 1 --2- 172.21.15.32:0/1212485202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68e4004d90 0x7f68e4005210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68f400e170 tx=0x7f68e4005880 comp rx=0 tx=0).stop 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f6900d6b700 1 -- 172.21.15.32:0/1212485202 reap_dead start 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f69027ce700 1 -- 172.21.15.32:0/1212485202 shutdown_connections 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f69027ce700 1 -- 172.21.15.32:0/1212485202 >> 172.21.15.32:0/1212485202 conn(0x7f68fc0cd570 msgr2=0x7f68fc0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f69027ce700 1 -- 172.21.15.32:0/1212485202 shutdown_connections 2023-11-24T07:59:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.412+0000 7f69027ce700 1 -- 172.21.15.32:0/1212485202 wait complete. 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.413+0000 7f69027ce700 1 Processor -- start 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 -- start start 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 0x7f68fc0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68fc155820 0x7f68fc155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc1564e0 0x7f68fc158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc156960 con 0x7f68fc14c5d0 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15f480 con 0x7f68fc155820 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.414+0000 7f69027ce700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b5030 con 0x7f68fc1564e0 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 0x7f68fc0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 0x7f68fc0b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57564/0 (socket says 172.21.15.32:57564) 2023-11-24T07:59:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 -- 172.21.15.32:0/435314924 learned_addr learned my addr 172.21.15.32:0/435314924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 -- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc1564e0 msgr2=0x7f68fc158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc1564e0 0x7f68fc158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 -- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68fc155820 msgr2=0x7f68fc155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68fc155820 0x7f68fc155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 -- 172.21.15.32:0/435314924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0b4a70 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68fbfff700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 0x7f68fc0b4120 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68ec011bb0 tx=0x7f68ec011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00fb10 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00fc70 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc0b4d50 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec004020 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.415+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc14c9b0 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.416+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc15f760 con 0x7f68fc14c5d0 2023-11-24T07:59:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.417+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68ec004180 con 0x7f68fc14c5d0 2023-11-24T07:59:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.417+0000 7f68f8ff9700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f68e806aed0 0x7f68e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.417+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68ec089c90 con 0x7f68fc14c5d0 2023-11-24T07:59:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.417+0000 7f68fb7fe700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f68e806aed0 0x7f68e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.418+0000 7f68fb7fe700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f68e806aed0 0x7f68e806d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0b5830 tx=0x7f68f0004870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.422+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68ec054d40 con 0x7f68fc14c5d0 2023-11-24T07:59:11.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.624+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68fc067f60 con 0x7f68fc14c5d0 2023-11-24T07:59:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.625+0000 7f68f8ff9700 1 -- 172.21.15.32:0/435314924 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f68ec022020 con 0x7f68fc14c5d0 2023-11-24T07:59:11.626 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:11.627 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:11.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f68e806aed0 msgr2=0x7f68e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f68e806aed0 0x7f68e806d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0b5830 tx=0x7f68f0004870 comp rx=0 tx=0).stop 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 msgr2=0x7f68fc0b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 0x7f68fc0b4120 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68ec011bb0 tx=0x7f68ec011f70 comp rx=0 tx=0).stop 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 shutdown_connections 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f68e806aed0 0x7f68e806d390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68fc1564e0 0x7f68fc158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68fc14c5d0 0x7f68fc0b4120 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 --2- 172.21.15.32:0/435314924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68fc155820 0x7f68fc155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 >> 172.21.15.32:0/435314924 conn(0x7f68fc0cd570 msgr2=0x7f68fc0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 shutdown_connections 2023-11-24T07:59:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.627+0000 7f69027ce700 1 -- 172.21.15.32:0/435314924 wait complete. 2023-11-24T07:59:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:11.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.767+0000 7f51ed89c700 1 Processor -- start 2023-11-24T07:59:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.767+0000 7f51ed89c700 1 -- start start 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51ed89c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51e815b370 0x7f51e815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51ed89c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e80b3c60 0x7f51e80b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51ed89c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80ae5e0 con 0x7f51e81587f0 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51ed89c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e80b4630 con 0x7f51e815b370 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51ed89c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e815be30 con 0x7f51e80b3c60 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51e815b370 0x7f51e815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51e815b370 0x7f51e815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57574/0 (socket says 172.21.15.32:57574) 2023-11-24T07:59:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 -- 172.21.15.32:0/1445921901 learned_addr learned my addr 172.21.15.32:0/1445921901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e77fe700 1 --2- 172.21.15.32:0/1445921901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e80b3c60 0x7f51e80b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 -- 172.21.15.32:0/1445921901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e80b3c60 msgr2=0x7f51e80b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 --2- 172.21.15.32:0/1445921901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e80b3c60 0x7f51e80b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 -- 172.21.15.32:0/1445921901 >> v1:172.21.15.123:6789/0 conn(0x7f51e81587f0 legacy=0x7f51e806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 -- 172.21.15.32:0/1445921901 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e815c110 con 0x7f51e815b370 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.769+0000 7f51e67fc700 1 --2- 172.21.15.32:0/1445921901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e815b800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009920 tx=0x7f51d8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1126e05ada0bbe8d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51e5ffb700 1 -- 172.21.15.32:0/1445921901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800ba20 con 0x7f51e815b370 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51e5ffb700 1 -- 172.21.15.32:0/1445921901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800bb80 con 0x7f51e815b370 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51e5ffb700 1 -- 172.21.15.32:0/1445921901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8009e70 con 0x7f51e815b370 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 -- 172.21.15.32:0/1445921901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 msgr2=0x7f51e815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 --2- 172.21.15.32:0/1445921901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e815b800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009920 tx=0x7f51d8009c30 comp rx=0 tx=0).stop 2023-11-24T07:59:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 -- 172.21.15.32:0/1445921901 shutdown_connections 2023-11-24T07:59:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 --2- 172.21.15.32:0/1445921901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e80b3c60 0x7f51e80b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 --2- 172.21.15.32:0/1445921901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e815b800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 -- 172.21.15.32:0/1445921901 >> 172.21.15.32:0/1445921901 conn(0x7f51e80d0bf0 msgr2=0x7f51e80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 -- 172.21.15.32:0/1445921901 shutdown_connections 2023-11-24T07:59:11.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.770+0000 7f51ed89c700 1 -- 172.21.15.32:0/1445921901 wait complete. 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.771+0000 7f51ed89c700 1 Processor -- start 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 -- start start 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e80b3c60 0x7f51e81f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e81587f0 0x7f51e8151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e8071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e8154750 con 0x7f51e815b370 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e8154c70 con 0x7f51e80b3c60 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51ed89c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81593c0 con 0x7f51e81587f0 2023-11-24T07:59:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e8071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e8071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57582/0 (socket says 172.21.15.32:57582) 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.772+0000 7f51e77fe700 1 -- 172.21.15.32:0/2926481826 learned_addr learned my addr 172.21.15.32:0/2926481826 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e67fc700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e81587f0 0x7f51e8151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e6ffd700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e80b3c60 0x7f51e81f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e77fe700 1 -- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e81587f0 msgr2=0x7f51e8151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e77fe700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e81587f0 0x7f51e8151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e77fe700 1 -- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e80b3c60 msgr2=0x7f51e81f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e77fe700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e80b3c60 0x7f51e81f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e77fe700 1 -- 172.21.15.32:0/2926481826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8009550 con 0x7f51e815b370 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e67fc700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e81587f0 0x7f51e8151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e6ffd700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e80b3c60 0x7f51e81f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51e77fe700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e8071bb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f51e0007e40 tx=0x7f51e000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e000ee40 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0014420 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00147d0 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e81f7cc0 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.773+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e8158d70 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.774+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e8158bd0 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.774+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51e0014930 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.775+0000 7f51cffff700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f51d406aed0 0x7f51d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.775+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51e0089d60 con 0x7f51e815b370 2023-11-24T07:59:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.775+0000 7f51e6ffd700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f51d406aed0 0x7f51d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.776+0000 7f51e6ffd700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f51d406aed0 0x7f51d406d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f60 tx=0x7f51dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.779+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51e8158bd0 con 0x7f51e815b370 2023-11-24T07:59:11.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.978+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51e8158bd0 con 0x7f51e815b370 2023-11-24T07:59:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.978+0000 7f51cffff700 1 -- 172.21.15.32:0/2926481826 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51e8158bd0 con 0x7f51e815b370 2023-11-24T07:59:11.980 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f51d406aed0 msgr2=0x7f51d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f51d406aed0 0x7f51d406d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f60 tx=0x7f51dc009040 comp rx=0 tx=0).stop 2023-11-24T07:59:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 msgr2=0x7f51e8071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e8071bb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f51e0007e40 tx=0x7f51e000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 shutdown_connections 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f51d406aed0 0x7f51d406d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e815b370 0x7f51e8071bb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e80b3c60 0x7f51e81f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 --2- 172.21.15.32:0/2926481826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51e81587f0 0x7f51e8151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.980+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 >> 172.21.15.32:0/2926481826 conn(0x7f51e80d0bf0 msgr2=0x7f51e80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.981+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 shutdown_connections 2023-11-24T07:59:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:11.981+0000 7f51ed89c700 1 -- 172.21.15.32:0/2926481826 wait complete. 2023-11-24T07:59:11.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.120+0000 7fcae8798700 1 Processor -- start 2023-11-24T07:59:12.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.121+0000 7fcae8798700 1 -- start start 2023-11-24T07:59:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae8798700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 0x7fcae006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae8798700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae8798700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae006b530 con 0x7fcae015b370 2023-11-24T07:59:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae8798700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae00ae5e0 con 0x7fcae01587f0 2023-11-24T07:59:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae8798700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae015be30 con 0x7fcae00b4370 2023-11-24T07:59:12.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae6534700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 0x7fcae006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae6d35700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae6534700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 0x7fcae006aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:41736/0 (socket says 172.21.15.32:41736) 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae5d33700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcae015b370 0x7fcae00b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:46052/0 (socket says 172.21.15.32:46052) 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.122+0000 7fcae6534700 1 -- 172.21.15.32:0/353264891 learned_addr learned my addr 172.21.15.32:0/353264891 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae6d35700 1 -- 172.21.15.32:0/353264891 >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 msgr2=0x7fcae006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae6d35700 1 --2- 172.21.15.32:0/353264891 >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 0x7fcae006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae6d35700 1 -- 172.21.15.32:0/353264891 >> v1:172.21.15.123:6789/0 conn(0x7fcae015b370 legacy=0x7fcae00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae6d35700 1 -- 172.21.15.32:0/353264891 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae015c110 con 0x7fcae00b4370 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae6534700 1 --2- 172.21.15.32:0/353264891 >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 0x7fcae006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:12.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae6d35700 1 --2- 172.21.15.32:0/353264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae015b800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcad800bfa0 tx=0x7fcad800d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8aeb99acb7fc907 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae5532700 1 -- 172.21.15.32:0/353264891 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8010070 con 0x7fcae00b4370 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae5532700 1 -- 172.21.15.32:0/353264891 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad800b2e0 con 0x7fcae00b4370 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.123+0000 7fcae5532700 1 -- 172.21.15.32:0/353264891 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800dc40 con 0x7fcae00b4370 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 -- 172.21.15.32:0/353264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 msgr2=0x7fcae015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 --2- 172.21.15.32:0/353264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae015b800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcad800bfa0 tx=0x7fcad800d330 comp rx=0 tx=0).stop 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 -- 172.21.15.32:0/353264891 shutdown_connections 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 --2- 172.21.15.32:0/353264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae015b800 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 --2- 172.21.15.32:0/353264891 >> v2:172.21.15.123:3300/0 conn(0x7fcae01587f0 0x7fcae006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 -- 172.21.15.32:0/353264891 >> 172.21.15.32:0/353264891 conn(0x7fcae00d0bf0 msgr2=0x7fcae00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 -- 172.21.15.32:0/353264891 shutdown_connections 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.124+0000 7fcae8798700 1 -- 172.21.15.32:0/353264891 wait complete. 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.125+0000 7fcae8798700 1 Processor -- start 2023-11-24T07:59:12.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.125+0000 7fcae8798700 1 -- start start 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae01540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 0x7fcae01465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015b370 0x7fcae006b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01ef820 con 0x7fcae01587f0 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0151600 con 0x7fcae015b370 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01efa60 con 0x7fcae00b4370 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 0x7fcae01465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 0x7fcae01465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57594/0 (socket says 172.21.15.32:57594) 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 -- 172.21.15.32:0/13360972 learned_addr learned my addr 172.21.15.32:0/13360972 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae6534700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae01540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae6d35700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015b370 0x7fcae006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 -- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 msgr2=0x7fcae01540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae01540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 -- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015b370 msgr2=0x7fcae006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015b370 0x7fcae006b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 -- 172.21.15.32:0/13360972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc009550 con 0x7fcae01587f0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae6534700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae01540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae5d33700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 0x7fcae01465f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c500 tx=0x7fcad400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4005cb0 con 0x7fcae01587f0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4005e10 con 0x7fcae01587f0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad800bc80 con 0x7fcae01587f0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.126+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40072d0 con 0x7fcae01587f0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.127+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae01f0020 con 0x7fcae01587f0 2023-11-24T07:59:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.127+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae01f01b0 con 0x7fcae01587f0 2023-11-24T07:59:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.128+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcad4007430 con 0x7fcae01587f0 2023-11-24T07:59:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.128+0000 7fcad2ffd700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac406aed0 0x7fcac406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.128+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcad4088910 con 0x7fcae01587f0 2023-11-24T07:59:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.128+0000 7fcae6534700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac406aed0 0x7fcac406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.129+0000 7fcae6534700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac406aed0 0x7fcac406d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcae00b25f0 tx=0x7fcadc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.133+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcad4053a70 con 0x7fcae01587f0 2023-11-24T07:59:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.333+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcae006e7c0 con 0x7fcae01587f0 2023-11-24T07:59:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.333+0000 7fcad2ffd700 1 -- 172.21.15.32:0/13360972 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcad40170a0 con 0x7fcae01587f0 2023-11-24T07:59:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac406aed0 msgr2=0x7fcac406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac406aed0 0x7fcac406d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcae00b25f0 tx=0x7fcadc008040 comp rx=0 tx=0).stop 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 msgr2=0x7fcae01465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 0x7fcae01465f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c500 tx=0x7fcad400c8c0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 shutdown_connections 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac406aed0 0x7fcac406d390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015b370 0x7fcae006b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b4370 0x7fcae01540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 --2- 172.21.15.32:0/13360972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae01587f0 0x7fcae01465f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 >> 172.21.15.32:0/13360972 conn(0x7fcae00d0bf0 msgr2=0x7fcae00aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 shutdown_connections 2023-11-24T07:59:12.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.336+0000 7fcae8798700 1 -- 172.21.15.32:0/13360972 wait complete. 2023-11-24T07:59:12.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.475+0000 7f13f84ab700 1 Processor -- start 2023-11-24T07:59:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.475+0000 7f13f84ab700 1 -- start start 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f84ab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f13f01587f0 0x7f13f006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f84ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 0x7f13f015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f84ab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f006b530 con 0x7f13f015b370 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f84ab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f00ae5e0 con 0x7f13f01587f0 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f84ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015be30 con 0x7f13f00b4370 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f6a48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 0x7f13f015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f6247700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f13f01587f0 0x7f13f006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f6a48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 0x7f13f015b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:52854/0 (socket says 172.21.15.32:52854) 2023-11-24T07:59:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.476+0000 7f13f6a48700 1 -- 172.21.15.32:0/1122630602 learned_addr learned my addr 172.21.15.32:0/1122630602 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f5245700 1 -- 172.21.15.32:0/1122630602 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688225902 0 0) 0x7f13f006b530 con 0x7f13f015b370 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f6a48700 1 -- 172.21.15.32:0/1122630602 >> v2:172.21.15.123:3300/0 conn(0x7f13f01587f0 msgr2=0x7f13f006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f6a48700 1 --2- 172.21.15.32:0/1122630602 >> v2:172.21.15.123:3300/0 conn(0x7f13f01587f0 0x7f13f006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f6a48700 1 -- 172.21.15.32:0/1122630602 >> v1:172.21.15.123:6789/0 conn(0x7f13f015b370 legacy=0x7f13f00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f6a48700 1 -- 172.21.15.32:0/1122630602 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f015c110 con 0x7f13f00b4370 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f6a48700 1 --2- 172.21.15.32:0/1122630602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 0x7f13f015b800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009e50 tx=0x7f13e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434cd81c46ce17b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f5245700 1 -- 172.21.15.32:0/1122630602 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800e070 con 0x7f13f00b4370 2023-11-24T07:59:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f5245700 1 -- 172.21.15.32:0/1122630602 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e800b910 con 0x7f13f00b4370 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.477+0000 7f13f5245700 1 -- 172.21.15.32:0/1122630602 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800bcc0 con 0x7f13f00b4370 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 -- 172.21.15.32:0/1122630602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 msgr2=0x7f13f015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 --2- 172.21.15.32:0/1122630602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 0x7f13f015b800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009e50 tx=0x7f13e800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 -- 172.21.15.32:0/1122630602 shutdown_connections 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 --2- 172.21.15.32:0/1122630602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f00b4370 0x7f13f015b800 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 --2- 172.21.15.32:0/1122630602 >> v2:172.21.15.123:3300/0 conn(0x7f13f01587f0 0x7f13f006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 -- 172.21.15.32:0/1122630602 >> 172.21.15.32:0/1122630602 conn(0x7f13f00d0bf0 msgr2=0x7f13f00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 -- 172.21.15.32:0/1122630602 shutdown_connections 2023-11-24T07:59:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.478+0000 7f13f84ab700 1 -- 172.21.15.32:0/1122630602 wait complete. 2023-11-24T07:59:12.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.479+0000 7f13f84ab700 1 Processor -- start 2023-11-24T07:59:12.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.479+0000 7f13f84ab700 1 -- start start 2023-11-24T07:59:12.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f84ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13f00b4370 0x7f13f009eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f84ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 0x7f13f01f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f84ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f015b370 0x7f13f01f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f84ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01f0fc0 con 0x7f13f01587f0 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f84ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01f12a0 con 0x7f13f00b4370 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f84ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01f3d90 con 0x7f13f015b370 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 0x7f13f01f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 0x7f13f01f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57606/0 (socket says 172.21.15.32:57606) 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 -- 172.21.15.32:0/946990593 learned_addr learned my addr 172.21.15.32:0/946990593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f6247700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13f00b4370 0x7f13f009eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f6a48700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f015b370 0x7f13f01f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 -- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f015b370 msgr2=0x7f13f01f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f015b370 0x7f13f01f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 -- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13f00b4370 msgr2=0x7f13f009eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13f00b4370 0x7f13f009eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.480+0000 7f13f5a46700 1 -- 172.21.15.32:0/946990593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8009b30 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.481+0000 7f13f5a46700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 0x7f13f01f4590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13ec001e70 tx=0x7f13ec00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.481+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0054e0 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.481+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec003680 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.481+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f01f4070 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.481+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec003a30 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.481+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f01f4ad0 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.482+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f0062190 con 0x7f13f01587f0 2023-11-24T07:59:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.482+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13ec003b90 con 0x7f13f01587f0 2023-11-24T07:59:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.482+0000 7f13e6ffd700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f13d406aed0 0x7f13d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.483+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13ec08afd0 con 0x7f13f01587f0 2023-11-24T07:59:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.483+0000 7f13f6247700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f13d406aed0 0x7f13d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.483+0000 7f13f6247700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f13d406aed0 0x7f13d406d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009be0 tx=0x7f13e0009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.487+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13ec055fe0 con 0x7f13f01587f0 2023-11-24T07:59:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.691+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13f01f4260 con 0x7f13f01587f0 2023-11-24T07:59:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.691+0000 7f13e6ffd700 1 -- 172.21.15.32:0/946990593 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13ec059600 con 0x7f13f01587f0 2023-11-24T07:59:12.693 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f13d406aed0 msgr2=0x7f13d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f13d406aed0 0x7f13d406d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009be0 tx=0x7f13e0009210 comp rx=0 tx=0).stop 2023-11-24T07:59:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 msgr2=0x7f13f01f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 0x7f13f01f4590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13ec001e70 tx=0x7f13ec00c800 comp rx=0 tx=0).stop 2023-11-24T07:59:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 shutdown_connections 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f13d406aed0 0x7f13d406d390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13f015b370 0x7f13f01f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13f00b4370 0x7f13f009eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 --2- 172.21.15.32:0/946990593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f01587f0 0x7f13f01f4590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 >> 172.21.15.32:0/946990593 conn(0x7f13f00d0bf0 msgr2=0x7f13f006b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 shutdown_connections 2023-11-24T07:59:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.693+0000 7f13f84ab700 1 -- 172.21.15.32:0/946990593 wait complete. 2023-11-24T07:59:12.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:12.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.831+0000 7f85ab3cb700 1 Processor -- start 2023-11-24T07:59:12.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.831+0000 7f85ab3cb700 1 -- start start 2023-11-24T07:59:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85ab3cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85a40adf00 0x7f85a406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85ab3cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a40af950 con 0x7f85a41446a0 2023-11-24T07:59:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85ab3cb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a40afc30 con 0x7f85a406c540 2023-11-24T07:59:12.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85ab3cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a406c9f0 con 0x7f85a40adf00 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85a40adf00 0x7f85a406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85a40adf00 0x7f85a406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57614/0 (socket says 172.21.15.32:57614) 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 -- 172.21.15.32:0/930681287 learned_addr learned my addr 172.21.15.32:0/930681287 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 -- 172.21.15.32:0/930681287 >> v1:172.21.15.123:6789/0 conn(0x7f85a406c540 legacy=0x7f85a40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 -- 172.21.15.32:0/930681287 >> v1:172.21.15.32:6789/0 conn(0x7f85a41446a0 legacy=0x7f85a40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 -- 172.21.15.32:0/930681287 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a40aff10 con 0x7f85a40adf00 2023-11-24T07:59:12.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f85a8966700 1 --2- 172.21.15.32:0/930681287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a40adf00 0x7f85a406c000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f85940098f0 tx=0x7f8594009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6974eb9ed05febbb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.832+0000 7f859bfff700 1 -- 172.21.15.32:0/930681287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400b8a0 con 0x7f85a40adf00 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f859bfff700 1 -- 172.21.15.32:0/930681287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859400ba00 con 0x7f85a40adf00 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f859bfff700 1 -- 172.21.15.32:0/930681287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400bd90 con 0x7f85a40adf00 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 -- 172.21.15.32:0/930681287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a40adf00 msgr2=0x7f85a406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/930681287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a40adf00 0x7f85a406c000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f85940098f0 tx=0x7f8594009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 -- 172.21.15.32:0/930681287 shutdown_connections 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/930681287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a40adf00 0x7f85a406c000 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 -- 172.21.15.32:0/930681287 >> 172.21.15.32:0/930681287 conn(0x7f85a4071b90 msgr2=0x7f85a4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 -- 172.21.15.32:0/930681287 shutdown_connections 2023-11-24T07:59:12.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.833+0000 7f85ab3cb700 1 -- 172.21.15.32:0/930681287 wait complete. 2023-11-24T07:59:12.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.834+0000 7f85ab3cb700 1 Processor -- start 2023-11-24T07:59:12.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.834+0000 7f85ab3cb700 1 -- start start 2023-11-24T07:59:12.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85ab3cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 0x7f85a4137460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85ab3cb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85a40adf00 0x7f85a415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85ab3cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85a41446a0 0x7f85a415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85ab3cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a413d8b0 con 0x7f85a406c540 2023-11-24T07:59:12.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85ab3cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a413db90 con 0x7f85a41446a0 2023-11-24T07:59:12.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85ab3cb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a413de70 con 0x7f85a40adf00 2023-11-24T07:59:12.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85a9167700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 0x7f85a4137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85a9167700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 0x7f85a4137460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57618/0 (socket says 172.21.15.32:57618) 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85a9167700 1 -- 172.21.15.32:0/1562188871 learned_addr learned my addr 172.21.15.32:0/1562188871 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85a8966700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85a40adf00 0x7f85a415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.835+0000 7f85a9968700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85a41446a0 0x7f85a415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85a9167700 1 -- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85a40adf00 msgr2=0x7f85a415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85a9167700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85a40adf00 0x7f85a415dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85a9167700 1 -- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85a41446a0 msgr2=0x7f85a415b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85a9167700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85a41446a0 0x7f85a415b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85a9167700 1 -- 172.21.15.32:0/1562188871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594009550 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85a9167700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 0x7f85a4137460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c450 tx=0x7f85a000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a00052d0 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0005430 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a00038c0 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a413e150 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.836+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a41379a0 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.837+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a40753f0 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.837+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85a0003a20 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.838+0000 7f8599ffb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f858806aed0 0x7f858806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.838+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85a00888a0 con 0x7f85a406c540 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.838+0000 7f85a8966700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f858806aed0 0x7f858806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.838+0000 7f85a8966700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f858806aed0 0x7f858806d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85940098f0 tx=0x7f8594008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:12.842+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85a00539e0 con 0x7f85a406c540 2023-11-24T07:59:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.059+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f85a415b310 con 0x7f85a406c540 2023-11-24T07:59:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.059+0000 7f8599ffb700 1 -- 172.21.15.32:0/1562188871 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f85a0057000 con 0x7f85a406c540 2023-11-24T07:59:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f858806aed0 msgr2=0x7f858806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f858806aed0 0x7f858806d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85940098f0 tx=0x7f8594008030 comp rx=0 tx=0).stop 2023-11-24T07:59:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 msgr2=0x7f85a4137460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 0x7f85a4137460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c450 tx=0x7f85a000c810 comp rx=0 tx=0).stop 2023-11-24T07:59:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 shutdown_connections 2023-11-24T07:59:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f858806aed0 0x7f858806d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85a41446a0 0x7f85a415b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85a406c540 0x7f85a4137460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 --2- 172.21.15.32:0/1562188871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85a40adf00 0x7f85a415dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 >> 172.21.15.32:0/1562188871 conn(0x7f85a4071b90 msgr2=0x7f85a415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 shutdown_connections 2023-11-24T07:59:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.061+0000 7f85ab3cb700 1 -- 172.21.15.32:0/1562188871 wait complete. 2023-11-24T07:59:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-11-24T07:59:13.075 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:59:13.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:13.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-24T07:59:13.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:59:13.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:13.127 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:59:13.151 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:13.151 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:13.173 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:13.173 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:13.234 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:13.235 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:13.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.419+0000 7faf4928b700 1 Processor -- start 2023-11-24T07:59:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.419+0000 7faf4928b700 1 -- start start 2023-11-24T07:59:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf4928b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf4415e1e0 0x7faf440d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf4928b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d0f80 0x7faf440b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf4928b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf440d20b0 con 0x7faf440d3a00 2023-11-24T07:59:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf4928b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf440d3150 con 0x7faf4415e1e0 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf4928b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4415e5c0 con 0x7faf440d0f80 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf42ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf4415e1e0 0x7faf440d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf427fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faf440d3a00 0x7faf440d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59340/0 (socket says 172.21.15.32:59340) 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf427fc700 1 -- 172.21.15.32:0/1081481050 learned_addr learned my addr 172.21.15.32:0/1081481050 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf42ffd700 1 -- 172.21.15.32:0/1081481050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d0f80 msgr2=0x7faf440b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf437fe700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d0f80 0x7faf440b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf42ffd700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d0f80 0x7faf440b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf42ffd700 1 -- 172.21.15.32:0/1081481050 >> v1:172.21.15.32:6789/0 conn(0x7faf440d3a00 legacy=0x7faf440d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf41ffb700 1 -- 172.21.15.32:0/1081481050 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269829439 0 0) 0x7faf440d20b0 con 0x7faf440d3a00 2023-11-24T07:59:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf42ffd700 1 -- 172.21.15.32:0/1081481050 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440d2390 con 0x7faf4415e1e0 2023-11-24T07:59:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.420+0000 7faf437fe700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d0f80 0x7faf440b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf42ffd700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf4415e1e0 0x7faf440d34c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf38009870 tx=0x7faf38009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9055b9f316f178d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf41ffb700 1 -- 172.21.15.32:0/1081481050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38011830 con 0x7faf4415e1e0 2023-11-24T07:59:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf41ffb700 1 -- 172.21.15.32:0/1081481050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38011990 con 0x7faf4415e1e0 2023-11-24T07:59:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf41ffb700 1 -- 172.21.15.32:0/1081481050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38011d20 con 0x7faf4415e1e0 2023-11-24T07:59:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 -- 172.21.15.32:0/1081481050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf4415e1e0 msgr2=0x7faf440d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf4415e1e0 0x7faf440d34c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf38009870 tx=0x7faf38009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 -- 172.21.15.32:0/1081481050 shutdown_connections 2023-11-24T07:59:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d0f80 0x7faf440b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 --2- 172.21.15.32:0/1081481050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf4415e1e0 0x7faf440d34c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 -- 172.21.15.32:0/1081481050 >> 172.21.15.32:0/1081481050 conn(0x7faf440b5b50 msgr2=0x7faf440b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 -- 172.21.15.32:0/1081481050 shutdown_connections 2023-11-24T07:59:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.421+0000 7faf4928b700 1 -- 172.21.15.32:0/1081481050 wait complete. 2023-11-24T07:59:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.423+0000 7faf4928b700 1 Processor -- start 2023-11-24T07:59:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.423+0000 7faf4928b700 1 -- start start 2023-11-24T07:59:13.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.423+0000 7faf4928b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 0x7faf4415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.423+0000 7faf4928b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d3a00 0x7faf440ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf4928b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4415e1e0 0x7faf44160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf4928b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4415ebe0 con 0x7faf440d0f80 2023-11-24T07:59:13.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf4928b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4415eec0 con 0x7faf4415e1e0 2023-11-24T07:59:13.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf4928b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4415f1a0 con 0x7faf440d3a00 2023-11-24T07:59:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 0x7faf4415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 0x7faf4415fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57632/0 (socket says 172.21.15.32:57632) 2023-11-24T07:59:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 -- 172.21.15.32:0/1405739216 learned_addr learned my addr 172.21.15.32:0/1405739216 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf427fc700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d3a00 0x7faf440ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf437fe700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4415e1e0 0x7faf44160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 -- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d3a00 msgr2=0x7faf440ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d3a00 0x7faf440ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 -- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4415e1e0 msgr2=0x7faf44160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4415e1e0 0x7faf44160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 -- 172.21.15.32:0/1405739216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380094a0 con 0x7faf440d0f80 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf42ffd700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 0x7faf4415fc80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf380090b0 tx=0x7faf38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380036a0 con 0x7faf440d0f80 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3801e070 con 0x7faf440d0f80 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4415f480 con 0x7faf440d0f80 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38003a40 con 0x7faf440d0f80 2023-11-24T07:59:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.424+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf440ca8f0 con 0x7faf440d0f80 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.425+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4415fad0 con 0x7faf440d0f80 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.426+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf38003ba0 con 0x7faf440d0f80 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.426+0000 7faf2b7fe700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7faf3006aed0 0x7faf3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.426+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf3808a5c0 con 0x7faf440d0f80 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.426+0000 7faf427fc700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7faf3006aed0 0x7faf3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.427+0000 7faf427fc700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7faf3006aed0 0x7faf3006d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf440cc720 tx=0x7faf34007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.431+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf38055700 con 0x7faf440d0f80 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.633+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf440751e0 con 0x7faf440d0f80 2023-11-24T07:59:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.634+0000 7faf2b7fe700 1 -- 172.21.15.32:0/1405739216 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7faf38058d20 con 0x7faf440d0f80 2023-11-24T07:59:13.637 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:13.637 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7faf3006aed0 msgr2=0x7faf3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7faf3006aed0 0x7faf3006d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf440cc720 tx=0x7faf34007c00 comp rx=0 tx=0).stop 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 msgr2=0x7faf4415fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 0x7faf4415fc80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf380090b0 tx=0x7faf38004750 comp rx=0 tx=0).stop 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 shutdown_connections 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7faf3006aed0 0x7faf3006d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4415e1e0 0x7faf44160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf440d0f80 0x7faf4415fc80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 --2- 172.21.15.32:0/1405739216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf440d3a00 0x7faf440ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 >> 172.21.15.32:0/1405739216 conn(0x7faf440b5b50 msgr2=0x7faf440ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 shutdown_connections 2023-11-24T07:59:13.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.636+0000 7faf4928b700 1 -- 172.21.15.32:0/1405739216 wait complete. 2023-11-24T07:59:13.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:13.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.777+0000 7f366f65b700 1 Processor -- start 2023-11-24T07:59:13.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.777+0000 7f366f65b700 1 -- start start 2023-11-24T07:59:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366f65b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f36680d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366f65b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f366806b700 0x7f3668167d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366f65b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680b6fb0 con 0x7f36680b93c0 2023-11-24T07:59:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366f65b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36680d4da0 con 0x7f366806b700 2023-11-24T07:59:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366f65b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680d2940 con 0x7f366815e320 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f366806b700 0x7f3668167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f366806b700 0x7f3668167d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57642/0 (socket says 172.21.15.32:57642) 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 -- 172.21.15.32:0/1959435632 learned_addr learned my addr 172.21.15.32:0/1959435632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366d3f7700 1 --2- 172.21.15.32:0/1959435632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f36680d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 -- 172.21.15.32:0/1959435632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 msgr2=0x7f36680d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 --2- 172.21.15.32:0/1959435632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f36680d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 -- 172.21.15.32:0/1959435632 >> v1:172.21.15.123:6789/0 conn(0x7f36680b93c0 legacy=0x7f366806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 -- 172.21.15.32:0/1959435632 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680d2c10 con 0x7f366806b700 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.778+0000 7f366dbf8700 1 --2- 172.21.15.32:0/1959435632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806b700 0x7f3668167d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3660009e50 tx=0x7f366000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db985af2709e3c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f365ffff700 1 -- 172.21.15.32:0/1959435632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000e070 con 0x7f366806b700 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f365ffff700 1 -- 172.21.15.32:0/1959435632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366000b9c0 con 0x7f366806b700 2023-11-24T07:59:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f365ffff700 1 -- 172.21.15.32:0/1959435632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000bd70 con 0x7f366806b700 2023-11-24T07:59:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 -- 172.21.15.32:0/1959435632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806b700 msgr2=0x7f3668167d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 --2- 172.21.15.32:0/1959435632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806b700 0x7f3668167d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3660009e50 tx=0x7f366000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 -- 172.21.15.32:0/1959435632 shutdown_connections 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 --2- 172.21.15.32:0/1959435632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806b700 0x7f3668167d70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 --2- 172.21.15.32:0/1959435632 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f36680d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 -- 172.21.15.32:0/1959435632 >> 172.21.15.32:0/1959435632 conn(0x7f36680d3e50 msgr2=0x7f36680d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 -- 172.21.15.32:0/1959435632 shutdown_connections 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.779+0000 7f366f65b700 1 -- 172.21.15.32:0/1959435632 wait complete. 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.781+0000 7f366f65b700 1 Processor -- start 2023-11-24T07:59:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.781+0000 7f366f65b700 1 -- start start 2023-11-24T07:59:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366f65b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366806b700 0x7f366814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366f65b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 0x7f3668158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366f65b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f3668159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366f65b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366815a7e0 con 0x7f36680b93c0 2023-11-24T07:59:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366f65b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668159b60 con 0x7f366806b700 2023-11-24T07:59:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366f65b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3668159e40 con 0x7f366815e320 2023-11-24T07:59:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 0x7f3668158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 0x7f3668158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57646/0 (socket says 172.21.15.32:57646) 2023-11-24T07:59:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 -- 172.21.15.32:0/1607501693 learned_addr learned my addr 172.21.15.32:0/1607501693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366d3f7700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366806b700 0x7f366814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366dbf8700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f3668159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 -- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 msgr2=0x7f3668159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f3668159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 -- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366806b700 msgr2=0x7f366814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366806b700 0x7f366814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366cbf6700 1 -- 172.21.15.32:0/1607501693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660009b30 con 0x7f36680b93c0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.782+0000 7f366dbf8700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f3668159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.783+0000 7f366cbf6700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 0x7f3668158ff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f365800c4b0 tx=0x7f365800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.783+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580052d0 con 0x7f36680b93c0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.783+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3658005430 con 0x7f36680b93c0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.783+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366815a120 con 0x7f36680b93c0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.783+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580038c0 con 0x7f36680b93c0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.783+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36681e6a30 con 0x7f36680b93c0 2023-11-24T07:59:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.784+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680cfc10 con 0x7f36680b93c0 2023-11-24T07:59:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.784+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3658003a20 con 0x7f36680b93c0 2023-11-24T07:59:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.785+0000 7f365dffb700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f364c06aed0 0x7f364c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.785+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36580889e0 con 0x7f36680b93c0 2023-11-24T07:59:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.785+0000 7f366d3f7700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f364c06aed0 0x7f364c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.785+0000 7f366d3f7700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f364c06aed0 0x7f364c06d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3668158d20 tx=0x7f3664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.789+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3658053a90 con 0x7f36680b93c0 2023-11-24T07:59:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.988+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f366815a920 con 0x7f36680b93c0 2023-11-24T07:59:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.989+0000 7f365dffb700 1 -- 172.21.15.32:0/1607501693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f366815a920 con 0x7f36680b93c0 2023-11-24T07:59:13.990 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f364c06aed0 msgr2=0x7f364c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f364c06aed0 0x7f364c06d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3668158d20 tx=0x7f3664008040 comp rx=0 tx=0).stop 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 msgr2=0x7f3668158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 0x7f3668158ff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f365800c4b0 tx=0x7f365800c7c0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 shutdown_connections 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f364c06aed0 0x7f364c06d390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366815e320 0x7f3668159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366806b700 0x7f366814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 --2- 172.21.15.32:0/1607501693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36680b93c0 0x7f3668158ff0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 >> 172.21.15.32:0/1607501693 conn(0x7f36680d3e50 msgr2=0x7f3668160e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 shutdown_connections 2023-11-24T07:59:13.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:13.991+0000 7f366f65b700 1 -- 172.21.15.32:0/1607501693 wait complete. 2023-11-24T07:59:14.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:14.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.132+0000 7f32309f1700 1 Processor -- start 2023-11-24T07:59:14.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.132+0000 7f32309f1700 1 -- start start 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f32309f1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f322c15e160 0x7f322c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f32309f1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f322c0afc90 0x7f322c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f32309f1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f32309f1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0b4500 con 0x7f322c0afc90 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f32309f1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c074960 con 0x7f322c15e160 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f32309f1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b3c60 con 0x7f322c0744d0 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f322c0afc90 0x7f322c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f322c0afc90 0x7f322c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57654/0 (socket says 172.21.15.32:57654) 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 -- 172.21.15.32:0/1538086123 learned_addr learned my addr 172.21.15.32:0/1538086123 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f322a59c700 1 --2- 172.21.15.32:0/1538086123 >> v2:172.21.15.123:3300/0 conn(0x7f322c15e160 0x7f322c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f322ad9d700 1 --2- 172.21.15.32:0/1538086123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 -- 172.21.15.32:0/1538086123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 msgr2=0x7f322c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 --2- 172.21.15.32:0/1538086123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 -- 172.21.15.32:0/1538086123 >> v2:172.21.15.123:3300/0 conn(0x7f322c15e160 msgr2=0x7f322c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 --2- 172.21.15.32:0/1538086123 >> v2:172.21.15.123:3300/0 conn(0x7f322c15e160 0x7f322c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 -- 172.21.15.32:0/1538086123 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0b58c0 con 0x7f322c0afc90 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.133+0000 7f3229d9b700 1 --2- 172.21.15.32:0/1538086123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c073f90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f322c09efa0 tx=0x7f321c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd5ca9c327d749a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f322959a700 1 -- 172.21.15.32:0/1538086123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c00b870 con 0x7f322c0afc90 2023-11-24T07:59:14.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f322959a700 1 -- 172.21.15.32:0/1538086123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c00b9d0 con 0x7f322c0afc90 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f322959a700 1 -- 172.21.15.32:0/1538086123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c00bd60 con 0x7f322c0afc90 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 -- 172.21.15.32:0/1538086123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 msgr2=0x7f322c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 --2- 172.21.15.32:0/1538086123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c073f90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f322c09efa0 tx=0x7f321c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 -- 172.21.15.32:0/1538086123 shutdown_connections 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 --2- 172.21.15.32:0/1538086123 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 --2- 172.21.15.32:0/1538086123 >> v2:172.21.15.123:3300/0 conn(0x7f322c15e160 0x7f322c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 --2- 172.21.15.32:0/1538086123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c073f90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 -- 172.21.15.32:0/1538086123 >> 172.21.15.32:0/1538086123 conn(0x7f322c0d0bf0 msgr2=0x7f322c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 -- 172.21.15.32:0/1538086123 shutdown_connections 2023-11-24T07:59:14.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.134+0000 7f32309f1700 1 -- 172.21.15.32:0/1538086123 wait complete. 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 Processor -- start 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 -- start start 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f322c15e160 0x7f322c154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1546f0 con 0x7f322c0afc90 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1f3a40 con 0x7f322c15e160 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f32309f1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1537d0 con 0x7f322c0744d0 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f3229d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f3229d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57660/0 (socket says 172.21.15.32:57660) 2023-11-24T07:59:14.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.136+0000 7f3229d9b700 1 -- 172.21.15.32:0/2845483558 learned_addr learned my addr 172.21.15.32:0/2845483558 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f322ad9d700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f322c15e160 0x7f322c154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f322a59c700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3229d9b700 1 -- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 msgr2=0x7f322c153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3229d9b700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3229d9b700 1 -- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f322c15e160 msgr2=0x7f322c154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3229d9b700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f322c15e160 0x7f322c154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3229d9b700 1 -- 172.21.15.32:0/2845483558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c009550 con 0x7f322c0afc90 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f322a59c700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3229d9b700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c14ee00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c004540 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c0036a0 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c153a70 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c003a50 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.137+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c154030 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.138+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c068fb0 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.138+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f321c021070 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.139+0000 7f3222ffd700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f320c06aed0 0x7f320c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.139+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f321c08ab40 con 0x7f322c0afc90 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.139+0000 7f322a59c700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f320c06aed0 0x7f320c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.140+0000 7f322a59c700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f320c06aed0 0x7f320c06d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32140079d0 tx=0x7f3214008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.143+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f321c055c80 con 0x7f322c0afc90 2023-11-24T07:59:14.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.346+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f322c06c140 con 0x7f322c0afc90 2023-11-24T07:59:14.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.346+0000 7f3222ffd700 1 -- 172.21.15.32:0/2845483558 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f321c015020 con 0x7f322c0afc90 2023-11-24T07:59:14.348 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f320c06aed0 msgr2=0x7f320c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f320c06aed0 0x7f320c06d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32140079d0 tx=0x7f3214008040 comp rx=0 tx=0).stop 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 msgr2=0x7f322c14ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c14ee00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 shutdown_connections 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f320c06aed0 0x7f320c06d390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f322c15e160 0x7f322c154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f322c0744d0 0x7f322c153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 --2- 172.21.15.32:0/2845483558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f322c0afc90 0x7f322c14ee00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 >> 172.21.15.32:0/2845483558 conn(0x7f322c0d0bf0 msgr2=0x7f322c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 shutdown_connections 2023-11-24T07:59:14.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.349+0000 7f32309f1700 1 -- 172.21.15.32:0/2845483558 wait complete. 2023-11-24T07:59:14.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.489+0000 7f047768f700 1 Processor -- start 2023-11-24T07:59:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.489+0000 7f047768f700 1 -- start start 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f047768f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04700b4370 0x7f047015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f047768f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04700ae5e0 con 0x7f04701587f0 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f047768f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047015be30 con 0x7f047015b370 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f047768f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047015c110 con 0x7f04700b4370 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0475c2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04700b4370 0x7f047015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f047542b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04701587f0 0x7f047006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59378/0 (socket says 172.21.15.32:59378) 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0475c2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04700b4370 0x7f047015b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57670/0 (socket says 172.21.15.32:57670) 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f047542b700 1 -- 172.21.15.32:0/2252766967 learned_addr learned my addr 172.21.15.32:0/2252766967 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0475c2c700 1 -- 172.21.15.32:0/2252766967 >> v1:172.21.15.123:6789/0 conn(0x7f047015b370 legacy=0x7f04700b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0475c2c700 1 -- 172.21.15.32:0/2252766967 >> v1:172.21.15.32:6789/0 conn(0x7f04701587f0 legacy=0x7f047006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0475c2c700 1 -- 172.21.15.32:0/2252766967 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04700ae860 con 0x7f04700b4370 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0467fff700 1 -- 172.21.15.32:0/2252766967 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025136525 0 0) 0x7f04700ae5e0 con 0x7f04701587f0 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.490+0000 7f0475c2c700 1 --2- 172.21.15.32:0/2252766967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f047015b800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0468009dc0 tx=0x7f046800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73554e5d7dc43e13 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f0467fff700 1 -- 172.21.15.32:0/2252766967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046800e070 con 0x7f04700b4370 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f0467fff700 1 -- 172.21.15.32:0/2252766967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046800bb50 con 0x7f04700b4370 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f0467fff700 1 -- 172.21.15.32:0/2252766967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468003a20 con 0x7f04700b4370 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 -- 172.21.15.32:0/2252766967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 msgr2=0x7f047015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 --2- 172.21.15.32:0/2252766967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f047015b800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0468009dc0 tx=0x7f046800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 -- 172.21.15.32:0/2252766967 shutdown_connections 2023-11-24T07:59:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 --2- 172.21.15.32:0/2252766967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f047015b800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 -- 172.21.15.32:0/2252766967 >> 172.21.15.32:0/2252766967 conn(0x7f04700d0bf0 msgr2=0x7f04700d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 -- 172.21.15.32:0/2252766967 shutdown_connections 2023-11-24T07:59:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.491+0000 7f047768f700 1 -- 172.21.15.32:0/2252766967 wait complete. 2023-11-24T07:59:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.492+0000 7f047768f700 1 Processor -- start 2023-11-24T07:59:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.492+0000 7f047768f700 1 -- start start 2023-11-24T07:59:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047768f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f04701f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047768f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04701587f0 0x7f047009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047768f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047015b370 0x7f04700b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047768f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047009bcd0 con 0x7f04700b4370 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047768f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047009b0c0 con 0x7f047015b370 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047768f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047009b3a0 con 0x7f04701587f0 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047542b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f04701f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047542b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f04701f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57672/0 (socket says 172.21.15.32:57672) 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f047542b700 1 -- 172.21.15.32:0/1280507896 learned_addr learned my addr 172.21.15.32:0/1280507896 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.493+0000 7f0475c2c700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047015b370 0x7f04700b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f0474c2a700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04701587f0 0x7f047009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047542b700 1 -- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04701587f0 msgr2=0x7f047009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047542b700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04701587f0 0x7f047009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047542b700 1 -- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047015b370 msgr2=0x7f04700b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047542b700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047015b370 0x7f04700b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047542b700 1 -- 172.21.15.32:0/1280507896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0468009a70 con 0x7f04700b4370 2023-11-24T07:59:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f0474c2a700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04701587f0 0x7f047009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047542b700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f04701f3bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f04600011e0 tx=0x7f046000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460004c80 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0460004e00 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047009b680 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460007310 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.494+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04701f6800 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.495+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0470068fb0 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.495+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0460007470 con 0x7f04700b4370 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.496+0000 7f0465ffb700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f045406aed0 0x7f045406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.496+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04600893b0 con 0x7f04700b4370 2023-11-24T07:59:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.496+0000 7f0474c2a700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f045406aed0 0x7f045406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.497+0000 7f0474c2a700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f045406aed0 0x7f045406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f046c006fd0 tx=0x7f046c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.500+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0460054460 con 0x7f04700b4370 2023-11-24T07:59:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.699+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f047006ee30 con 0x7f04700b4370 2023-11-24T07:59:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.699+0000 7f0465ffb700 1 -- 172.21.15.32:0/1280507896 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0460057a80 con 0x7f04700b4370 2023-11-24T07:59:14.701 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f045406aed0 msgr2=0x7f045406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f045406aed0 0x7f045406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f046c006fd0 tx=0x7f046c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 msgr2=0x7f04701f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f04701f3bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f04600011e0 tx=0x7f046000cae0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 shutdown_connections 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f045406aed0 0x7f045406d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047015b370 0x7f04700b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04700b4370 0x7f04701f3bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 --2- 172.21.15.32:0/1280507896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04701587f0 0x7f047009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 >> 172.21.15.32:0/1280507896 conn(0x7f04700d0bf0 msgr2=0x7f047006f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 shutdown_connections 2023-11-24T07:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.702+0000 7f047768f700 1 -- 172.21.15.32:0/1280507896 wait complete. 2023-11-24T07:59:14.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.842+0000 7fe8eb46f700 1 Processor -- start 2023-11-24T07:59:14.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.842+0000 7fe8eb46f700 1 -- start start 2023-11-24T07:59:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8eb46f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 0x7fe8e406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8eb46f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e415c4b0 con 0x7fe8e40d0630 2023-11-24T07:59:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8eb46f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e415c790 con 0x7fe8e40b49a0 2023-11-24T07:59:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8eb46f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e415ca70 con 0x7fe8e4152cc0 2023-11-24T07:59:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8e9a0c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8e40d0630 0x7fe8e415fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59396/0 (socket says 172.21.15.32:59396) 2023-11-24T07:59:14.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8e9a0c700 1 -- 172.21.15.32:0/4039554785 learned_addr learned my addr 172.21.15.32:0/4039554785 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8e920b700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 0x7fe8e406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048624007 0 0) 0x7fe8e415c4b0 con 0x7fe8e40d0630 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d00035b0 con 0x7fe8e40d0630 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091009022 0 0) 0x7fe8e415c790 con 0x7fe8e40b49a0 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e415c4b0 con 0x7fe8e40b49a0 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55695495 0 0) 0x7fe8d00035b0 con 0x7fe8e40d0630 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 msgr2=0x7fe8e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 0x7fe8e406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 >> v1:172.21.15.123:6789/0 conn(0x7fe8e40b49a0 legacy=0x7fe8e40cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e415cd50 con 0x7fe8e40d0630 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.843+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fe8dc0019f0 con 0x7fe8e40d0630 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 >> v1:172.21.15.32:6789/0 conn(0x7fe8e40d0630 legacy=0x7fe8e415fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0003bc0 0x7fe8d0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d0004850 0x7fe8d0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 0x7fe8d0005310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e415ca70 con 0x7fe8d0004850 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d000bc10 con 0x7fe8d0003bc0 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d000bdd0 con 0x7fe8e4152cc0 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d0004850 0x7fe8d0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e920b700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0003bc0 0x7fe8d0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e9a0c700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 0x7fe8d0005310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 msgr2=0x7fe8d0005310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8e4152cc0 0x7fe8d0005310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0003bc0 msgr2=0x7fe8d0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8d0003bc0 0x7fe8d0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/4039554785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d00035b0 con 0x7fe8d0004850 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d0004850 0x7fe8d0004cd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4001940 tx=0x7fe8d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e0f5e104dca3f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4001430 con 0x7fe8d0004850 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4001d70 con 0x7fe8d0004850 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.844+0000 7fe8dbfff700 1 -- 172.21.15.32:0/4039554785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4005510 con 0x7fe8d0004850 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8eb46f700 1 -- 172.21.15.32:0/4039554785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d0004850 msgr2=0x7fe8d0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/4039554785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8d0004850 0x7fe8d0004cd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4001940 tx=0x7fe8d0005880 comp rx=0 tx=0).stop 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8e9a0c700 1 -- 172.21.15.32:0/4039554785 reap_dead start 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8eb46f700 1 -- 172.21.15.32:0/4039554785 shutdown_connections 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8eb46f700 1 -- 172.21.15.32:0/4039554785 >> 172.21.15.32:0/4039554785 conn(0x7fe8e4071b90 msgr2=0x7fe8e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8eb46f700 1 -- 172.21.15.32:0/4039554785 shutdown_connections 2023-11-24T07:59:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.845+0000 7fe8eb46f700 1 -- 172.21.15.32:0/4039554785 wait complete. 2023-11-24T07:59:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.847+0000 7fe8eb46f700 1 Processor -- start 2023-11-24T07:59:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.847+0000 7fe8eb46f700 1 -- start start 2023-11-24T07:59:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8d0004850 0x7fe8e40b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 0x7fe8e415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40b0670 0x7fe8e4147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e40b17b0 con 0x7fe8e4149070 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e40b0c30 con 0x7fe8e40b0670 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e40b0f10 con 0x7fe8d0004850 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 0x7fe8e415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 0x7fe8e415e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57694/0 (socket says 172.21.15.32:57694) 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/1144081869 learned_addr learned my addr 172.21.15.32:0/1144081869 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e9a0c700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40b0670 0x7fe8e4147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e920b700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8d0004850 0x7fe8e40b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8d0004850 msgr2=0x7fe8e40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8d0004850 0x7fe8e40b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40b0670 msgr2=0x7fe8e4147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40b0670 0x7fe8e4147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 -- 172.21.15.32:0/1144081869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40b11f0 con 0x7fe8e4149070 2023-11-24T07:59:14.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e9a0c700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40b0670 0x7fe8e4147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e920b700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8d0004850 0x7fe8e40b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8e8a0a700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 0x7fe8e415e6c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4004000 tx=0x7fe8d40012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4006c70 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4006dd0 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e40b14d0 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.848+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4007280 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.849+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e415ec00 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.850+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e415edf0 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.850+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8d40073e0 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.851+0000 7fe8d9ffb700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8c806aed0 0x7fe8c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.851+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8d408a410 con 0x7fe8e4149070 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.851+0000 7fe8e920b700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8c806aed0 0x7fe8c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.851+0000 7fe8e920b700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8c806aed0 0x7fe8c806d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006dd0 tx=0x7fe8e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:14.855+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8d4055550 con 0x7fe8e4149070 2023-11-24T07:59:15.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.071+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8e40753f0 con 0x7fe8e4149070 2023-11-24T07:59:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.071+0000 7fe8d9ffb700 1 -- 172.21.15.32:0/1144081869 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe8d4058b70 con 0x7fe8e4149070 2023-11-24T07:59:15.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8c806aed0 msgr2=0x7fe8c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8c806aed0 0x7fe8c806d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0006dd0 tx=0x7fe8e0008040 comp rx=0 tx=0).stop 2023-11-24T07:59:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 msgr2=0x7fe8e415e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 0x7fe8e415e6c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4004000 tx=0x7fe8d40012d0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 shutdown_connections 2023-11-24T07:59:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8c806aed0 0x7fe8c806d390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40b0670 0x7fe8e4147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8d0004850 0x7fe8e40b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 --2- 172.21.15.32:0/1144081869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8e4149070 0x7fe8e415e6c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 >> 172.21.15.32:0/1144081869 conn(0x7fe8e4071b90 msgr2=0x7fe8e4151820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 shutdown_connections 2023-11-24T07:59:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.073+0000 7fe8eb46f700 1 -- 172.21.15.32:0/1144081869 wait complete. 2023-11-24T07:59:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-11-24T07:59:15.088 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:59:15.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:15.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-24T07:59:15.135 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:59:15.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:15.136 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:59:15.167 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:15.167 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:15.184 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:15.185 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:15.242 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:15.242 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:15.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.464+0000 7f9ac8f2d700 1 Processor -- start 2023-11-24T07:59:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.464+0000 7f9ac8f2d700 1 -- start start 2023-11-24T07:59:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac8f2d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac8f2d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ac406d350 0x7f9ac40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac8f2d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac40cc5c0 con 0x7f9ac415de50 2023-11-24T07:59:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac8f2d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac4071d70 con 0x7f9ac406d350 2023-11-24T07:59:15.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac8f2d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40b3070 con 0x7f9ac4151500 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac1d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ac406d350 0x7f9ac40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac259c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac2d9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9ac415de50 0x7f9ac40b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:46170/0 (socket says 172.21.15.32:46170) 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.465+0000 7f9ac2d9d700 1 -- 172.21.15.32:0/1018141872 learned_addr learned my addr 172.21.15.32:0/1018141872 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac259c700 1 -- 172.21.15.32:0/1018141872 >> v2:172.21.15.123:3300/0 conn(0x7f9ac406d350 msgr2=0x7f9ac40cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac259c700 1 --2- 172.21.15.32:0/1018141872 >> v2:172.21.15.123:3300/0 conn(0x7f9ac406d350 0x7f9ac40cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac259c700 1 -- 172.21.15.32:0/1018141872 >> v1:172.21.15.123:6789/0 conn(0x7f9ac415de50 legacy=0x7f9ac40b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac259c700 1 -- 172.21.15.32:0/1018141872 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40cf6a0 con 0x7f9ac4151500 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac1d9b700 1 --2- 172.21.15.32:0/1018141872 >> v2:172.21.15.123:3300/0 conn(0x7f9ac406d350 0x7f9ac40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac259c700 1 --2- 172.21.15.32:0/1018141872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b49b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009840 tx=0x7f9ab4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c774b908628912f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac159a700 1 -- 172.21.15.32:0/1018141872 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4011830 con 0x7f9ac4151500 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac159a700 1 -- 172.21.15.32:0/1018141872 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4011990 con 0x7f9ac4151500 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.466+0000 7f9ac159a700 1 -- 172.21.15.32:0/1018141872 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4011d20 con 0x7f9ac4151500 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/1018141872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 msgr2=0x7f9ac40b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/1018141872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b49b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009840 tx=0x7f9ab4009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/1018141872 shutdown_connections 2023-11-24T07:59:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/1018141872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b49b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/1018141872 >> v2:172.21.15.123:3300/0 conn(0x7f9ac406d350 0x7f9ac40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/1018141872 >> 172.21.15.32:0/1018141872 conn(0x7f9ac40b1740 msgr2=0x7f9ac40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/1018141872 shutdown_connections 2023-11-24T07:59:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.467+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/1018141872 wait complete. 2023-11-24T07:59:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.468+0000 7f9ac8f2d700 1 Processor -- start 2023-11-24T07:59:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.468+0000 7f9ac8f2d700 1 -- start start 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac8f2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 0x7f9ac415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac8f2d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac8f2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac415de50 0x7f9ac4137030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac8f2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4149c00 con 0x7f9ac406d350 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac8f2d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4148ff0 con 0x7f9ac415de50 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac8f2d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41492d0 con 0x7f9ac4151500 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 0x7f9ac415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 0x7f9ac415aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57706/0 (socket says 172.21.15.32:57706) 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 -- 172.21.15.32:0/3600816957 learned_addr learned my addr 172.21.15.32:0/3600816957 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac2d9d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac415de50 0x7f9ac4137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac1d9b700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 -- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 msgr2=0x7f9ac40b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 -- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac415de50 msgr2=0x7f9ac4137030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac415de50 0x7f9ac4137030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac259c700 1 -- 172.21.15.32:0/3600816957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80094a0 con 0x7f9ac406d350 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac2d9d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac415de50 0x7f9ac4137030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.469+0000 7f9ac1d9b700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.470+0000 7f9ac259c700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 0x7f9ac415aa60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009fd0 tx=0x7f9ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.470+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40036a0 con 0x7f9ac406d350 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.470+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab401f070 con 0x7f9ac406d350 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.470+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4003a60 con 0x7f9ac406d350 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.470+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab40094a0 con 0x7f9ac406d350 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.470+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac4149890 con 0x7f9ac406d350 2023-11-24T07:59:15.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.471+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40751e0 con 0x7f9ac406d350 2023-11-24T07:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.471+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ab4003bc0 con 0x7f9ac406d350 2023-11-24T07:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.472+0000 7f9ab2ffd700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9aa406aed0 0x7f9aa406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.472+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab4021070 con 0x7f9ac406d350 2023-11-24T07:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.472+0000 7f9ac1d9b700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9aa406aed0 0x7f9aa406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.472+0000 7f9ac1d9b700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9aa406aed0 0x7f9aa406d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9ac41f5730 tx=0x7f9ab8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.476+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ab4055fd0 con 0x7f9ac406d350 2023-11-24T07:59:15.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.679+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ac4149d40 con 0x7f9ac406d350 2023-11-24T07:59:15.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.680+0000 7f9ab2ffd700 1 -- 172.21.15.32:0/3600816957 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f9ac4149d40 con 0x7f9ac406d350 2023-11-24T07:59:15.683 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:15.683 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:15.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9aa406aed0 msgr2=0x7f9aa406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9aa406aed0 0x7f9aa406d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9ac41f5730 tx=0x7f9ab8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 msgr2=0x7f9ac415aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 0x7f9ac415aa60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009fd0 tx=0x7f9ab4004750 comp rx=0 tx=0).stop 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 shutdown_connections 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9aa406aed0 0x7f9aa406d390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ac415de50 0x7f9ac4137030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ac406d350 0x7f9ac415aa60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 --2- 172.21.15.32:0/3600816957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ac4151500 0x7f9ac40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 >> 172.21.15.32:0/3600816957 conn(0x7f9ac40b1740 msgr2=0x7f9ac40cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 shutdown_connections 2023-11-24T07:59:15.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.683+0000 7f9ac8f2d700 1 -- 172.21.15.32:0/3600816957 wait complete. 2023-11-24T07:59:15.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:15.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:15.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.825+0000 7fe9144e5700 1 Processor -- start 2023-11-24T07:59:15.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.825+0000 7fe9144e5700 1 -- start start 2023-11-24T07:59:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe9144e5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 0x7fe90c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe9144e5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe90c073f90 0x7fe90c074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe9144e5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c06b080 con 0x7fe90c0cf610 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe9144e5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c074960 con 0x7fe90c073f90 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe9144e5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c15c290 con 0x7fe90c15f9e0 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe912281700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe90c0cf610 0x7fe90c0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59426/0 (socket says 172.21.15.32:59426) 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe912281700 1 -- 172.21.15.32:0/3065996073 learned_addr learned my addr 172.21.15.32:0/3065996073 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe911a80700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 0x7fe90c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe912a82700 1 --2- 172.21.15.32:0/3065996073 >> v2:172.21.15.123:3300/0 conn(0x7fe90c073f90 0x7fe90c074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502683681 0 0) 0x7fe90c06b080 con 0x7fe90c0cf610 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f80035b0 con 0x7fe90c0cf610 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4158355862 0 0) 0x7fe8f80035b0 con 0x7fe90c0cf610 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 msgr2=0x7fe90c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 0x7fe90c0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 >> v2:172.21.15.123:3300/0 conn(0x7fe90c073f90 msgr2=0x7fe90c074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 --2- 172.21.15.32:0/3065996073 >> v2:172.21.15.123:3300/0 conn(0x7fe90c073f90 0x7fe90c074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0cf9f0 con 0x7fe90c0cf610 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fe908001670 con 0x7fe90c0cf610 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe912a82700 1 --2- 172.21.15.32:0/3065996073 >> v2:172.21.15.123:3300/0 conn(0x7fe90c073f90 0x7fe90c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 >> v1:172.21.15.32:6789/0 conn(0x7fe90c0cf610 legacy=0x7fe90c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.826+0000 7fe91127f700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8f8003a00 0x7fe8f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f8004850 0x7fe8f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 0x7fe8f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c074960 con 0x7fe8f8004850 2023-11-24T07:59:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c15c290 con 0x7fe8f8003a00 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f800ba10 con 0x7fe90c15f9e0 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f8004850 0x7fe8f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe912281700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8f8003a00 0x7fe8f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe912a82700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 0x7fe8f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 -- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 msgr2=0x7fe8f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe90c15f9e0 0x7fe8f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 -- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8f8003a00 msgr2=0x7fe8f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8f8003a00 0x7fe8f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 -- 172.21.15.32:0/3065996073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f80035b0 con 0x7fe8f8004850 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe911a80700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f8004850 0x7fe8f8004cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009b60 tx=0x7fe8f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4116740bcde0b72f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00e6c0 con 0x7fe8f8004850 2023-11-24T07:59:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00e820 con 0x7fe8f8004850 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.827+0000 7fe91127f700 1 -- 172.21.15.32:0/3065996073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00eb90 con 0x7fe8f8004850 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe9144e5700 1 -- 172.21.15.32:0/3065996073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f8004850 msgr2=0x7fe8f8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe9144e5700 1 --2- 172.21.15.32:0/3065996073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f8004850 0x7fe8f8004cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009b60 tx=0x7fe8f8005880 comp rx=0 tx=0).stop 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe912a82700 1 -- 172.21.15.32:0/3065996073 reap_dead start 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe9144e5700 1 -- 172.21.15.32:0/3065996073 shutdown_connections 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe9144e5700 1 -- 172.21.15.32:0/3065996073 >> 172.21.15.32:0/3065996073 conn(0x7fe90c0d0bf0 msgr2=0x7fe90c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe9144e5700 1 -- 172.21.15.32:0/3065996073 shutdown_connections 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.828+0000 7fe9144e5700 1 -- 172.21.15.32:0/3065996073 wait complete. 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.829+0000 7fe9144e5700 1 Processor -- start 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 -- start start 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8f8004850 0x7fe90c14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 0x7fe90c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe90c15a510 0x7fe90c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c15d390 con 0x7fe90c150cc0 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c15aed0 con 0x7fe90c15a510 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.830+0000 7fe9144e5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c1548d0 con 0x7fe8f8004850 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 0x7fe90c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 0x7fe90c14d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57728/0 (socket says 172.21.15.32:57728) 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 -- 172.21.15.32:0/689461764 learned_addr learned my addr 172.21.15.32:0/689461764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe912281700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8f8004850 0x7fe90c14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe912a82700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe90c15a510 0x7fe90c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 -- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8f8004850 msgr2=0x7fe90c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8f8004850 0x7fe90c14db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 -- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe90c15a510 msgr2=0x7fe90c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe90c15a510 0x7fe90c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 -- 172.21.15.32:0/689461764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c154fa0 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe911a80700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 0x7fe90c14d3a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e4b0 tx=0x7fe8fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc021070 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc004460 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c151140 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00e6c0 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.831+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c155210 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.832+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c15d600 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.833+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8fc025070 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.833+0000 7fe902ffd700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8f006aed0 0x7fe8f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.833+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8fc08a490 con 0x7fe90c150cc0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.833+0000 7fe912281700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8f006aed0 0x7fe8f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.834+0000 7fe912281700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8f006aed0 0x7fe8f006d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe90c1f5580 tx=0x7fe908007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:15.838+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8fc0556f0 con 0x7fe90c150cc0 2023-11-24T07:59:16.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.036+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe90c068fb0 con 0x7fe90c150cc0 2023-11-24T07:59:16.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.037+0000 7fe902ffd700 1 -- 172.21.15.32:0/689461764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8fc058d10 con 0x7fe90c150cc0 2023-11-24T07:59:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8f006aed0 msgr2=0x7fe8f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8f006aed0 0x7fe8f006d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe90c1f5580 tx=0x7fe908007980 comp rx=0 tx=0).stop 2023-11-24T07:59:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 msgr2=0x7fe90c14d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 0x7fe90c14d3a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e4b0 tx=0x7fe8fc0047c0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 shutdown_connections 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe8f006aed0 0x7fe8f006d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe90c15a510 0x7fe90c15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe8f8004850 0x7fe90c14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 --2- 172.21.15.32:0/689461764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe90c150cc0 0x7fe90c14d3a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 >> 172.21.15.32:0/689461764 conn(0x7fe90c0d0bf0 msgr2=0x7fe90c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 shutdown_connections 2023-11-24T07:59:16.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.039+0000 7fe9144e5700 1 -- 172.21.15.32:0/689461764 wait complete. 2023-11-24T07:59:16.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.179+0000 7f07a6598700 1 Processor -- start 2023-11-24T07:59:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.180+0000 7f07a6598700 1 -- start start 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a6598700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07a015ada0 0x7f07a006f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a6598700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f07a00afb40 0x7f07a00cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a6598700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a00cfa80 0x7f07a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a6598700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00cd2e0 con 0x7f07a015ada0 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a6598700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00cd5c0 con 0x7f07a00afb40 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a6598700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cccd0 con 0x7f07a00cfa80 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07a015ada0 0x7f07a006f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07a015ada0 0x7f07a006f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57734/0 (socket says 172.21.15.32:57734) 2023-11-24T07:59:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 -- 172.21.15.32:0/3798986167 learned_addr learned my addr 172.21.15.32:0/3798986167 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079f7fe700 1 --2- 172.21.15.32:0/3798986167 >> v2:172.21.15.123:3300/0 conn(0x7f07a00afb40 0x7f07a00cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 -- 172.21.15.32:0/3798986167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a00cfa80 msgr2=0x7f07a006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a4b35700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a00cfa80 0x7f07a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a00cfa80 0x7f07a006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 -- 172.21.15.32:0/3798986167 >> v2:172.21.15.123:3300/0 conn(0x7f07a00afb40 msgr2=0x7f07a00cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 --2- 172.21.15.32:0/3798986167 >> v2:172.21.15.123:3300/0 conn(0x7f07a00afb40 0x7f07a00cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 -- 172.21.15.32:0/3798986167 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a006fd50 con 0x7f07a015ada0 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079f7fe700 1 --2- 172.21.15.32:0/3798986167 >> v2:172.21.15.123:3300/0 conn(0x7f07a00afb40 0x7f07a00cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f07a4b35700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a00cfa80 0x7f07a006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079ffff700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a015ada0 0x7f07a006f810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07a006c010 tx=0x7f0790009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12099d9fa27082c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.181+0000 7f079effd700 1 -- 172.21.15.32:0/3798986167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000b870 con 0x7f07a015ada0 2023-11-24T07:59:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f079effd700 1 -- 172.21.15.32:0/3798986167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000b9d0 con 0x7f07a015ada0 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f079effd700 1 -- 172.21.15.32:0/3798986167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000bd60 con 0x7f07a015ada0 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 -- 172.21.15.32:0/3798986167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a015ada0 msgr2=0x7f07a006f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a015ada0 0x7f07a006f810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f07a006c010 tx=0x7f0790009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 -- 172.21.15.32:0/3798986167 shutdown_connections 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a00cfa80 0x7f07a006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 --2- 172.21.15.32:0/3798986167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a015ada0 0x7f07a006f810 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 --2- 172.21.15.32:0/3798986167 >> v2:172.21.15.123:3300/0 conn(0x7f07a00afb40 0x7f07a00cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 -- 172.21.15.32:0/3798986167 >> 172.21.15.32:0/3798986167 conn(0x7f07a00b2050 msgr2=0x7f07a00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 -- 172.21.15.32:0/3798986167 shutdown_connections 2023-11-24T07:59:16.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.182+0000 7f07a6598700 1 -- 172.21.15.32:0/3798986167 wait complete. 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.183+0000 7f07a6598700 1 Processor -- start 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 -- start start 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 0x7f07a014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a00cfa80 0x7f07a00b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a015ada0 0x7f07a01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01f7990 con 0x7f07a00afb40 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01f7e80 con 0x7f07a00cfa80 2023-11-24T07:59:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f07a6598700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0151e50 con 0x7f07a015ada0 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f079ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 0x7f07a014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f079ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 0x7f07a014cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57740/0 (socket says 172.21.15.32:57740) 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.184+0000 7f079ffff700 1 -- 172.21.15.32:0/3046958824 learned_addr learned my addr 172.21.15.32:0/3046958824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079f7fe700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a00cfa80 0x7f07a00b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f07a4b35700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a015ada0 0x7f07a01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079ffff700 1 -- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a015ada0 msgr2=0x7f07a01593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079ffff700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a015ada0 0x7f07a01593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079ffff700 1 -- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a00cfa80 msgr2=0x7f07a00b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079ffff700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a00cfa80 0x7f07a00b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079ffff700 1 -- 172.21.15.32:0/3046958824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790009550 con 0x7f07a00afb40 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079f7fe700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a00cfa80 0x7f07a00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f07a4b35700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a015ada0 0x7f07a01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079ffff700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 0x7f07a014cfe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0790014040 tx=0x7f0790004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790004520 con 0x7f07a00afb40 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079001e070 con 0x7f07a00afb40 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a01f7ba0 con 0x7f07a00afb40 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07900038e0 con 0x7f07a00afb40 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.185+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a00b1580 con 0x7f07a00afb40 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.186+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a0151ca0 con 0x7f07a00afb40 2023-11-24T07:59:16.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.186+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0790003a40 con 0x7f07a00afb40 2023-11-24T07:59:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.187+0000 7f079cff9700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f078c06aed0 0x7f078c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.187+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0790020070 con 0x7f07a00afb40 2023-11-24T07:59:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.187+0000 7f079f7fe700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f078c06aed0 0x7f078c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.187+0000 7f079f7fe700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f078c06aed0 0x7f078c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0794006fd0 tx=0x7f0794008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.191+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0790055da0 con 0x7f07a00afb40 2023-11-24T07:59:16.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.393+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07a0068d90 con 0x7f07a00afb40 2023-11-24T07:59:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.393+0000 7f079cff9700 1 -- 172.21.15.32:0/3046958824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f079002a0a0 con 0x7f07a00afb40 2023-11-24T07:59:16.395 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.395+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f078c06aed0 msgr2=0x7f078c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f078c06aed0 0x7f078c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0794006fd0 tx=0x7f0794008040 comp rx=0 tx=0).stop 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 msgr2=0x7f07a014cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 0x7f07a014cfe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0790014040 tx=0x7f0790004750 comp rx=0 tx=0).stop 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 shutdown_connections 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f078c06aed0 0x7f078c06d390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07a015ada0 0x7f07a01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07a00afb40 0x7f07a014cfe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 --2- 172.21.15.32:0/3046958824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a00cfa80 0x7f07a00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 >> 172.21.15.32:0/3046958824 conn(0x7f07a00b2050 msgr2=0x7f07a015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 shutdown_connections 2023-11-24T07:59:16.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.396+0000 7f07a6598700 1 -- 172.21.15.32:0/3046958824 wait complete. 2023-11-24T07:59:16.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.536+0000 7f3de0d4d700 1 Processor -- start 2023-11-24T07:59:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.536+0000 7f3de0d4d700 1 -- start start 2023-11-24T07:59:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.537+0000 7f3de0d4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.537+0000 7f3de0d4d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ddc0b4370 0x7f3ddc15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.537+0000 7f3de0d4d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc06b530 con 0x7f3ddc15b370 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.537+0000 7f3de0d4d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc0ae5e0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.537+0000 7f3de0d4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc15be30 con 0x7f3ddc1587f0 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ddc0b4370 0x7f3ddc15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ddc0b4370 0x7f3ddc15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57752/0 (socket says 172.21.15.32:57752) 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 -- 172.21.15.32:0/233689935 learned_addr learned my addr 172.21.15.32:0/233689935 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3dd9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ddc15b370 0x7f3ddc0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59462/0 (socket says 172.21.15.32:59462) 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3dda59c700 1 --2- 172.21.15.32:0/233689935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 -- 172.21.15.32:0/233689935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 msgr2=0x7f3ddc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 --2- 172.21.15.32:0/233689935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3dd959a700 1 -- 172.21.15.32:0/233689935 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831582828 0 0) 0x7f3ddc06b530 con 0x7f3ddc15b370 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 -- 172.21.15.32:0/233689935 >> v1:172.21.15.32:6789/0 conn(0x7f3ddc15b370 legacy=0x7f3ddc0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 -- 172.21.15.32:0/233689935 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc15c110 con 0x7f3ddc0b4370 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3ddad9d700 1 --2- 172.21.15.32:0/233689935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc15b7f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009e50 tx=0x7f3dd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ddb182d1edb902 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3dd959a700 1 -- 172.21.15.32:0/233689935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400e070 con 0x7f3ddc0b4370 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3dd959a700 1 -- 172.21.15.32:0/233689935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400b910 con 0x7f3ddc0b4370 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.538+0000 7f3dd959a700 1 -- 172.21.15.32:0/233689935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bcc0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 -- 172.21.15.32:0/233689935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 msgr2=0x7f3ddc15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/233689935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc15b7f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009e50 tx=0x7f3dd400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 -- 172.21.15.32:0/233689935 shutdown_connections 2023-11-24T07:59:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/233689935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc15b7f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/233689935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 -- 172.21.15.32:0/233689935 >> 172.21.15.32:0/233689935 conn(0x7f3ddc0d0bf0 msgr2=0x7f3ddc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 -- 172.21.15.32:0/233689935 shutdown_connections 2023-11-24T07:59:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.539+0000 7f3de0d4d700 1 -- 172.21.15.32:0/233689935 wait complete. 2023-11-24T07:59:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.540+0000 7f3de0d4d700 1 Processor -- start 2023-11-24T07:59:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.540+0000 7f3de0d4d700 1 -- start start 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3de0d4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc1475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3de0d4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3de0d4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc15b370 0x7f3ddc155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3de0d4d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b13d0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3de0d4d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc147b20 con 0x7f3ddc15b370 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3de0d4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b07e0 con 0x7f3ddc1587f0 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc1475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc1475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57754/0 (socket says 172.21.15.32:57754) 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 -- 172.21.15.32:0/3879519813 learned_addr learned my addr 172.21.15.32:0/3879519813 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3ddad9d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc15b370 0x7f3ddc155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dd9d9b700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 -- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 msgr2=0x7f3ddc0b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc0b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 -- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc15b370 msgr2=0x7f3ddc155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc15b370 0x7f3ddc155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.541+0000 7f3dda59c700 1 -- 172.21.15.32:0/3879519813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009b30 con 0x7f3ddc0b4370 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3ddad9d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc15b370 0x7f3ddc155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3dd9d9b700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3dda59c700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc1475e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e9b0 tx=0x7f3dc400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4009710 con 0x7f3ddc0b4370 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400c850 con 0x7f3ddc0b4370 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc0b0a30 con 0x7f3ddc0b4370 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400cbc0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.542+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc0b0ff0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.543+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc06aff0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.543+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3dc400cd20 con 0x7f3ddc0b4370 2023-11-24T07:59:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.544+0000 7f3dd2ffd700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3dbc06aed0 0x7f3dbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.544+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dc40896e0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.544+0000 7f3dd9d9b700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3dbc06aed0 0x7f3dbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.544+0000 7f3dd9d9b700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3dbc06aed0 0x7f3dbc06d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc007fd0 tx=0x7f3dcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.548+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3dc4054780 con 0x7f3ddc0b4370 2023-11-24T07:59:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.750+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ddc0b1640 con 0x7f3ddc0b4370 2023-11-24T07:59:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.750+0000 7f3dd2ffd700 1 -- 172.21.15.32:0/3879519813 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3dc4057da0 con 0x7f3ddc0b4370 2023-11-24T07:59:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:16.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3dbc06aed0 msgr2=0x7f3dbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3dbc06aed0 0x7f3dbc06d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc007fd0 tx=0x7f3dcc009040 comp rx=0 tx=0).stop 2023-11-24T07:59:16.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 msgr2=0x7f3ddc1475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc1475e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e9b0 tx=0x7f3dc400ed70 comp rx=0 tx=0).stop 2023-11-24T07:59:16.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 shutdown_connections 2023-11-24T07:59:16.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3dbc06aed0 0x7f3dbc06d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc15b370 0x7f3ddc155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc0b4370 0x7f3ddc1475e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 --2- 172.21.15.32:0/3879519813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ddc1587f0 0x7f3ddc0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 >> 172.21.15.32:0/3879519813 conn(0x7f3ddc0d0bf0 msgr2=0x7f3ddc0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 shutdown_connections 2023-11-24T07:59:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.752+0000 7f3de0d4d700 1 -- 172.21.15.32:0/3879519813 wait complete. 2023-11-24T07:59:16.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.892+0000 7f44ee863700 1 Processor -- start 2023-11-24T07:59:16.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.892+0000 7f44ee863700 1 -- start start 2023-11-24T07:59:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44ee863700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e81446a0 0x7f44e80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44ee863700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e80af950 con 0x7f44e80adf00 2023-11-24T07:59:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44ee863700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e80afc30 con 0x7f44e806c710 2023-11-24T07:59:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44ee863700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8144a80 con 0x7f44e81446a0 2023-11-24T07:59:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44e80adf00 0x7f44e806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59476/0 (socket says 172.21.15.32:59476) 2023-11-24T07:59:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e77fe700 1 -- 172.21.15.32:0/1118292138 learned_addr learned my addr 172.21.15.32:0/1118292138 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e7fff700 1 --2- 172.21.15.32:0/1118292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e81446a0 0x7f44e80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373747327 0 0) 0x7f44e80af950 con 0x7f44e80adf00 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d00035b0 con 0x7f44e80adf00 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572088422 0 0) 0x7f44e80afc30 con 0x7f44e806c710 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.893+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e80af950 con 0x7f44e806c710 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e7fff700 1 -- 172.21.15.32:0/1118292138 >> v1:172.21.15.123:6789/0 conn(0x7f44e806c710 legacy=0x7f44e80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e7fff700 1 -- 172.21.15.32:0/1118292138 >> v1:172.21.15.32:6789/0 conn(0x7f44e80adf00 legacy=0x7f44e806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e7fff700 1 -- 172.21.15.32:0/1118292138 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80aff10 con 0x7f44e81446a0 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e7fff700 1 --2- 172.21.15.32:0/1118292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e81446a0 0x7f44e80b5ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009870 tx=0x7f44d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551ac8fa4e97a5ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8011830 con 0x7f44e81446a0 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d8011990 con 0x7f44e81446a0 2023-11-24T07:59:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.894+0000 7f44e6ffd700 1 -- 172.21.15.32:0/1118292138 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8011d20 con 0x7f44e81446a0 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 -- 172.21.15.32:0/1118292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e81446a0 msgr2=0x7f44e80b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 --2- 172.21.15.32:0/1118292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e81446a0 0x7f44e80b5ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009870 tx=0x7f44d8009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 -- 172.21.15.32:0/1118292138 shutdown_connections 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 --2- 172.21.15.32:0/1118292138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e81446a0 0x7f44e80b5ae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 -- 172.21.15.32:0/1118292138 >> 172.21.15.32:0/1118292138 conn(0x7f44e8071b90 msgr2=0x7f44e8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 -- 172.21.15.32:0/1118292138 shutdown_connections 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.895+0000 7f44ee863700 1 -- 172.21.15.32:0/1118292138 wait complete. 2023-11-24T07:59:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.896+0000 7f44ee863700 1 Processor -- start 2023-11-24T07:59:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.896+0000 7f44ee863700 1 -- start start 2023-11-24T07:59:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ee863700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e806c710 0x7f44e81efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ee863700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 0x7f44e814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ee863700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e81446a0 0x7f44e814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ee863700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81f00e0 con 0x7f44e80adf00 2023-11-24T07:59:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ee863700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81f02d0 con 0x7f44e81446a0 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ee863700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81f05b0 con 0x7f44e806c710 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 0x7f44e814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 0x7f44e814f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57770/0 (socket says 172.21.15.32:57770) 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 -- 172.21.15.32:0/2093467133 learned_addr learned my addr 172.21.15.32:0/2093467133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44ece00700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e81446a0 0x7f44e814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e7fff700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e806c710 0x7f44e81efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 -- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e806c710 msgr2=0x7f44e81efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e806c710 0x7f44e81efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 -- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e81446a0 msgr2=0x7f44e814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e81446a0 0x7f44e814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 -- 172.21.15.32:0/2093467133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d80094a0 con 0x7f44e80adf00 2023-11-24T07:59:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e77fe700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 0x7f44e814f000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005910 tx=0x7f44dc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.897+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc005d70 con 0x7f44e80adf00 2023-11-24T07:59:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.898+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc0070d0 con 0x7f44e80adf00 2023-11-24T07:59:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.898+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81f0890 con 0x7f44e80adf00 2023-11-24T07:59:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.898+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc007460 con 0x7f44e80adf00 2023-11-24T07:59:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.898+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81f0e50 con 0x7f44e80adf00 2023-11-24T07:59:16.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.899+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e81f1010 con 0x7f44e80adf00 2023-11-24T07:59:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.899+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44dc0075c0 con 0x7f44e80adf00 2023-11-24T07:59:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.900+0000 7f44e4ff9700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f44d406aed0 0x7f44d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.900+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44dc088b90 con 0x7f44e80adf00 2023-11-24T07:59:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.900+0000 7f44e7fff700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f44d406aed0 0x7f44d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.900+0000 7f44e7fff700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f44d406aed0 0x7f44d406d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f44d80097c0 tx=0x7f44d801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:16.904+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44dc053c40 con 0x7f44e80adf00 2023-11-24T07:59:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.120+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44e80753f0 con 0x7f44e80adf00 2023-11-24T07:59:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.120+0000 7f44e4ff9700 1 -- 172.21.15.32:0/2093467133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44dc017080 con 0x7f44e80adf00 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f44d406aed0 msgr2=0x7f44d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f44d406aed0 0x7f44d406d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f44d80097c0 tx=0x7f44d801c040 comp rx=0 tx=0).stop 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 msgr2=0x7f44e814f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 0x7f44e814f000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005910 tx=0x7f44dc00ca90 comp rx=0 tx=0).stop 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 shutdown_connections 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f44d406aed0 0x7f44d406d390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44e81446a0 0x7f44e814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44e806c710 0x7f44e81efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 --2- 172.21.15.32:0/2093467133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e80adf00 0x7f44e814f000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 >> 172.21.15.32:0/2093467133 conn(0x7f44e8071b90 msgr2=0x7f44e80b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 shutdown_connections 2023-11-24T07:59:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.122+0000 7f44ee863700 1 -- 172.21.15.32:0/2093467133 wait complete. 2023-11-24T07:59:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-11-24T07:59:17.136 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-24T07:59:17.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:17.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-24T07:59:17.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:59:17.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:17.174 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T07:59:17.232 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:17.233 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:17.255 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:17.255 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:17.318 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:17.319 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:17.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.502+0000 7fdd00fb4700 1 Processor -- start 2023-11-24T07:59:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.502+0000 7fdd00fb4700 1 -- start start 2023-11-24T07:59:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.503+0000 7fdd00fb4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcfc0d2380 0x7fdcfc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.503+0000 7fdd00fb4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc0b22d0 con 0x7fdcfc0b3370 2023-11-24T07:59:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.503+0000 7fdd00fb4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc0b25b0 con 0x7fdcfc15bdd0 2023-11-24T07:59:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.503+0000 7fdd00fb4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b2f50 con 0x7fdcfc0d2380 2023-11-24T07:59:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.503+0000 7fdcf9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdcfc0b3370 0x7fdcfc0d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59490/0 (socket says 172.21.15.32:59490) 2023-11-24T07:59:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.503+0000 7fdcf9d9b700 1 -- 172.21.15.32:0/3425829375 learned_addr learned my addr 172.21.15.32:0/3425829375 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:17.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487066550 0 0) 0x7fdcfc0b22d0 con 0x7fdcfc0b3370 2023-11-24T07:59:17.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcfad9d700 1 --2- 172.21.15.32:0/3425829375 >> v2:172.21.15.123:3300/0 conn(0x7fdcfc0d2380 0x7fdcfc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce80035b0 con 0x7fdcfc0b3370 2023-11-24T07:59:17.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2919923067 0 0) 0x7fdce80035b0 con 0x7fdcfc0b3370 2023-11-24T07:59:17.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 >> v2:172.21.15.123:3300/0 conn(0x7fdcfc0d2380 msgr2=0x7fdcfc1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:17.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 --2- 172.21.15.32:0/3425829375 >> v2:172.21.15.123:3300/0 conn(0x7fdcfc0d2380 0x7fdcfc1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 >> v1:172.21.15.123:6789/0 conn(0x7fdcfc15bdd0 legacy=0x7fdcfc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0b2890 con 0x7fdcfc0b3370 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fdcec001670 con 0x7fdcfc0b3370 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 >> v1:172.21.15.32:6789/0 conn(0x7fdcfc0b3370 legacy=0x7fdcfc0d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce8003a00 0x7fdce8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8004850 0x7fdce8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8005210 0x7fdce800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0b2f50 con 0x7fdce8003a00 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce800c0b0 con 0x7fdce8005210 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce800c270 con 0x7fdce8004850 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce8003a00 0x7fdce8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcfad9d700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8005210 0x7fdce800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.504+0000 7fdcf9d9b700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8004850 0x7fdce8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfa59c700 1 -- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8004850 msgr2=0x7fdce8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdce8004850 0x7fdce8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfa59c700 1 -- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8005210 msgr2=0x7fdce800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8005210 0x7fdce800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfad9d700 1 -- 172.21.15.32:0/3425829375 reap_dead start 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfa59c700 1 -- 172.21.15.32:0/3425829375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80035b0 con 0x7fdce8003a00 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce8003a00 0x7fdce8003e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdce40017c0 tx=0x7fdce8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d70c3f036dd14df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40052d0 con 0x7fdce8003a00 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4005450 con 0x7fdce8003a00 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdcf959a700 1 -- 172.21.15.32:0/3425829375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40038c0 con 0x7fdce8003a00 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3425829375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce8003a00 msgr2=0x7fdce8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.505+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce8003a00 0x7fdce8003e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdce40017c0 tx=0x7fdce8005880 comp rx=0 tx=0).stop 2023-11-24T07:59:17.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.506+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3425829375 shutdown_connections 2023-11-24T07:59:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.506+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3425829375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce8003a00 0x7fdce8003e20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.506+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3425829375 >> 172.21.15.32:0/3425829375 conn(0x7fdcfc0731a0 msgr2=0x7fdcfc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.506+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3425829375 shutdown_connections 2023-11-24T07:59:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.506+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3425829375 wait complete. 2023-11-24T07:59:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.507+0000 7fdd00fb4700 1 Processor -- start 2023-11-24T07:59:17.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.507+0000 7fdd00fb4700 1 -- start start 2023-11-24T07:59:17.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdd00fb4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8003a00 0x7fdcfc06ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdd00fb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 0x7fdcfc157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdd00fb4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcfc0cd6d0 0x7fdcfc0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdd00fb4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0c4850 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdd00fb4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc0ce090 con 0x7fdce8003a00 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdd00fb4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc06bf40 con 0x7fdcfc0cd6d0 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 0x7fdcfc157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 0x7fdcfc157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57788/0 (socket says 172.21.15.32:57788) 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 -- 172.21.15.32:0/3837190233 learned_addr learned my addr 172.21.15.32:0/3837190233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8003a00 0x7fdcfc06ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcfad9d700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcfc0cd6d0 0x7fdcfc0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 -- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcfc0cd6d0 msgr2=0x7fdcfc0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcfc0cd6d0 0x7fdcfc0cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 -- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8003a00 msgr2=0x7fdcfc06ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8003a00 0x7fdcfc06ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 -- 172.21.15.32:0/3837190233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc157ac0 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf9d9b700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 0x7fdcfc157580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0072c0 tx=0x7fdcec007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.508+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0077a0 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.509+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00f850 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.509+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc06c170 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.509+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00fc00 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.509+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc06c730 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.509+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc067f60 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.510+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcec00fd60 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.510+0000 7fdcf2ffd700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fdcdc06aed0 0x7fdcdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.511+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcec089510 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.511+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fdcdc06aed0 0x7fdcdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.511+0000 7fdcfa59c700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fdcdc06aed0 0x7fdcdc06d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdce4000c00 tx=0x7fdce4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.515+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdcec054670 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.721+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcfc0b7c60 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.722+0000 7fdcf2ffd700 1 -- 172.21.15.32:0/3837190233 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fdcec087e60 con 0x7fdcfc0c4c20 2023-11-24T07:59:17.724 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:17.724 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fdcdc06aed0 msgr2=0x7fdcdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fdcdc06aed0 0x7fdcdc06d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdce4000c00 tx=0x7fdce4009030 comp rx=0 tx=0).stop 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 msgr2=0x7fdcfc157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 0x7fdcfc157580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0072c0 tx=0x7fdcec007680 comp rx=0 tx=0).stop 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 shutdown_connections 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fdcdc06aed0 0x7fdcdc06d390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcfc0cd6d0 0x7fdcfc0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdce8003a00 0x7fdcfc06ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 --2- 172.21.15.32:0/3837190233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcfc0c4c20 0x7fdcfc157580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 >> 172.21.15.32:0/3837190233 conn(0x7fdcfc0731a0 msgr2=0x7fdcfc0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 shutdown_connections 2023-11-24T07:59:17.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.724+0000 7fdd00fb4700 1 -- 172.21.15.32:0/3837190233 wait complete. 2023-11-24T07:59:17.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:17.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.864+0000 7f7f50cb4700 1 Processor -- start 2023-11-24T07:59:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.864+0000 7f7f50cb4700 1 -- start start 2023-11-24T07:59:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f50cb4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f4c0afb40 0x7f7f4c0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f50cb4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c0cfa80 0x7f7f4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f50cb4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c0cccd0 con 0x7f7f4c15ada0 2023-11-24T07:59:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f50cb4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4c0cd2e0 con 0x7f7f4c0afb40 2023-11-24T07:59:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f50cb4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0cd5c0 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f49d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f4c0afb40 0x7f7f4c0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f4a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f4c15ada0 0x7f7f4c06f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59506/0 (socket says 172.21.15.32:59506) 2023-11-24T07:59:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f49d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f4c0afb40 0x7f7f4c0cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57796/0 (socket says 172.21.15.32:57796) 2023-11-24T07:59:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.865+0000 7f7f4a59c700 1 -- 172.21.15.32:0/709713289 learned_addr learned my addr 172.21.15.32:0/709713289 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f4ad9d700 1 --2- 172.21.15.32:0/709713289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c0cfa80 0x7f7f4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f49d9b700 1 -- 172.21.15.32:0/709713289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c0cfa80 msgr2=0x7f7f4c06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f49d9b700 1 --2- 172.21.15.32:0/709713289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c0cfa80 0x7f7f4c06d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f49d9b700 1 -- 172.21.15.32:0/709713289 >> v1:172.21.15.32:6789/0 conn(0x7f7f4c15ada0 legacy=0x7f7f4c06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f49d9b700 1 -- 172.21.15.32:0/709713289 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c062440 con 0x7f7f4c0afb40 2023-11-24T07:59:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f4959a700 1 -- 172.21.15.32:0/709713289 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37637691 0 0) 0x7f7f4c0cccd0 con 0x7f7f4c15ada0 2023-11-24T07:59:17.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f49d9b700 1 --2- 172.21.15.32:0/709713289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c0cf540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00bae0 tx=0x7f7f3c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca1df6f3c221ac2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f4959a700 1 -- 172.21.15.32:0/709713289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00dd30 con 0x7f7f4c0afb40 2023-11-24T07:59:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f4959a700 1 -- 172.21.15.32:0/709713289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00de90 con 0x7f7f4c0afb40 2023-11-24T07:59:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.866+0000 7f7f4959a700 1 -- 172.21.15.32:0/709713289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c010690 con 0x7f7f4c0afb40 2023-11-24T07:59:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 -- 172.21.15.32:0/709713289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0afb40 msgr2=0x7f7f4c0cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/709713289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c0cf540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00bae0 tx=0x7f7f3c00bdf0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 -- 172.21.15.32:0/709713289 shutdown_connections 2023-11-24T07:59:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/709713289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c0cfa80 0x7f7f4c06d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/709713289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c0cf540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 -- 172.21.15.32:0/709713289 >> 172.21.15.32:0/709713289 conn(0x7f7f4c0b2050 msgr2=0x7f7f4c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 -- 172.21.15.32:0/709713289 shutdown_connections 2023-11-24T07:59:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.867+0000 7f7f50cb4700 1 -- 172.21.15.32:0/709713289 wait complete. 2023-11-24T07:59:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.868+0000 7f7f50cb4700 1 Processor -- start 2023-11-24T07:59:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.868+0000 7f7f50cb4700 1 -- start start 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f50cb4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f50cb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 0x7f7f4c13e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f50cb4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c15ada0 0x7f7f4c13e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f50cb4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c150340 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f50cb4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1553f0 con 0x7f7f4c0afb40 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f50cb4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1556d0 con 0x7f7f4c15ada0 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 0x7f7f4c13e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 0x7f7f4c13e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57802/0 (socket says 172.21.15.32:57802) 2023-11-24T07:59:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 -- 172.21.15.32:0/3388884986 learned_addr learned my addr 172.21.15.32:0/3388884986 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f4a59c700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f4ad9d700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c15ada0 0x7f7f4c13e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 -- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c15ada0 msgr2=0x7f7f4c13e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c15ada0 0x7f7f4c13e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 -- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c0afb40 msgr2=0x7f7f4c154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 -- 172.21.15.32:0/3388884986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c00b740 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.869+0000 7f7f49d9b700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 0x7f7f4c13e040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.870+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01f070 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.870+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c003970 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.870+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c1559b0 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.870+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c003d40 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.870+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c155f70 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.871+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c068d90 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.871+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f3c02d430 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.872+0000 7f7f3affd700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7f2c06aed0 0x7f7f2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.872+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f3c08ca60 con 0x7f7f4c0cfa80 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.872+0000 7f7f4a59c700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7f2c06aed0 0x7f7f2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.873+0000 7f7f4a59c700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7f2c06aed0 0x7f7f2c06d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:17.876+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f3c057c40 con 0x7f7f4c0cfa80 2023-11-24T07:59:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.074+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f4c156100 con 0x7f7f4c0cfa80 2023-11-24T07:59:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.075+0000 7f7f3affd700 1 -- 172.21.15.32:0/3388884986 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f3c05b260 con 0x7f7f4c0cfa80 2023-11-24T07:59:18.077 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7f2c06aed0 msgr2=0x7f7f2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7f2c06aed0 0x7f7f2c06d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40007e70 comp rx=0 tx=0).stop 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 msgr2=0x7f7f4c13e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 0x7f7f4c13e040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000c00 tx=0x7f7f3c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 shutdown_connections 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7f2c06aed0 0x7f7f2c06d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f4c15ada0 0x7f7f4c13e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c0afb40 0x7f7f4c154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 --2- 172.21.15.32:0/3388884986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f4c0cfa80 0x7f7f4c13e040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 >> 172.21.15.32:0/3388884986 conn(0x7f7f4c0b2050 msgr2=0x7f7f4c150d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 shutdown_connections 2023-11-24T07:59:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.077+0000 7f7f50cb4700 1 -- 172.21.15.32:0/3388884986 wait complete. 2023-11-24T07:59:18.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.216+0000 7fa4a8c79700 1 Processor -- start 2023-11-24T07:59:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.216+0000 7fa4a8c79700 1 -- start start 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.217+0000 7fa4a8c79700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a4140540 0x7fa4a4140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.217+0000 7fa4a8c79700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a406de00 con 0x7fa4a406e290 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.217+0000 7fa4a8c79700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40d0070 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.217+0000 7fa4a8c79700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40d0bb0 con 0x7fa4a4140540 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.217+0000 7fa4a2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa4a406e290 0x7fa4a406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59524/0 (socket says 172.21.15.32:59524) 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.217+0000 7fa4a2d9d700 1 -- 172.21.15.32:0/1391279793 learned_addr learned my addr 172.21.15.32:0/1391279793 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a259c700 1 --2- 172.21.15.32:0/1391279793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a4140540 0x7fa4a4140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415107957 0 0) 0x7fa4a406de00 con 0x7fa4a406e290 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa48c0035b0 con 0x7fa4a406e290 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434544461 0 0) 0x7fa4a40d0070 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4a406de00 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a259c700 1 -- 172.21.15.32:0/1391279793 >> v1:172.21.15.123:6789/0 conn(0x7fa4a40cf4e0 legacy=0x7fa4a40cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a259c700 1 -- 172.21.15.32:0/1391279793 >> v1:172.21.15.32:6789/0 conn(0x7fa4a406e290 legacy=0x7fa4a406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a259c700 1 -- 172.21.15.32:0/1391279793 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4140e60 con 0x7fa4a4140540 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a259c700 1 --2- 172.21.15.32:0/1391279793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a4140540 0x7fa4a4140920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa498009870 tx=0x7fa498009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9653652020df4fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011830 con 0x7fa4a4140540 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498011990 con 0x7fa4a4140540 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.218+0000 7fa4a159a700 1 -- 172.21.15.32:0/1391279793 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011d20 con 0x7fa4a4140540 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 -- 172.21.15.32:0/1391279793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a4140540 msgr2=0x7fa4a4140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/1391279793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a4140540 0x7fa4a4140920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa498009870 tx=0x7fa498009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 -- 172.21.15.32:0/1391279793 shutdown_connections 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/1391279793 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a4140540 0x7fa4a4140920 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 -- 172.21.15.32:0/1391279793 >> 172.21.15.32:0/1391279793 conn(0x7fa4a4072960 msgr2=0x7fa4a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 -- 172.21.15.32:0/1391279793 shutdown_connections 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.219+0000 7fa4a8c79700 1 -- 172.21.15.32:0/1391279793 wait complete. 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.221+0000 7fa4a8c79700 1 Processor -- start 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.221+0000 7fa4a8c79700 1 -- start start 2023-11-24T07:59:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a8c79700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a406e290 0x7fa4a415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a8c79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 0x7fa4a406b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a8c79700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a4140540 0x7fa4a406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a8c79700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a415e710 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a8c79700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a415cf90 con 0x7fa4a4140540 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a8c79700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a415c660 con 0x7fa4a406e290 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 0x7fa4a406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 0x7fa4a406b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57816/0 (socket says 172.21.15.32:57816) 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 -- 172.21.15.32:0/2987501781 learned_addr learned my addr 172.21.15.32:0/2987501781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a259c700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a406e290 0x7fa4a415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a2d9d700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a4140540 0x7fa4a406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 -- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a406e290 msgr2=0x7fa4a415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a406e290 0x7fa4a415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 -- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a4140540 msgr2=0x7fa4a406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a4140540 0x7fa4a406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 -- 172.21.15.32:0/2987501781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980094a0 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a2d9d700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a4140540 0x7fa4a406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa4a1d9b700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 0x7fa4a406b060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c550 tx=0x7fa49400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940052d0 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494005430 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.222+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940038c0 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.223+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a415c940 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.223+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a415d420 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.223+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a4068fb0 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.224+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa494003a20 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.224+0000 7fa492ffd700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa48406aed0 0x7fa48406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.224+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa494088a20 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.225+0000 7fa4a259c700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa48406aed0 0x7fa48406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.225+0000 7fa4a259c700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa48406aed0 0x7fa48406d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4980097c0 tx=0x7fa49801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.229+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa494053b60 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.430+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4a406f710 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.430+0000 7fa492ffd700 1 -- 172.21.15.32:0/2987501781 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa494017020 con 0x7fa4a40cf4e0 2023-11-24T07:59:18.432 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:18.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.432+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa48406aed0 msgr2=0x7fa48406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.432+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa48406aed0 0x7fa48406d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4980097c0 tx=0x7fa49801c040 comp rx=0 tx=0).stop 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.432+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 msgr2=0x7fa4a406b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.432+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 0x7fa4a406b060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c550 tx=0x7fa49400c860 comp rx=0 tx=0).stop 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 shutdown_connections 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa48406aed0 0x7fa48406d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a4140540 0x7fa4a406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a406e290 0x7fa4a415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 --2- 172.21.15.32:0/2987501781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4a40cf4e0 0x7fa4a406b060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 >> 172.21.15.32:0/2987501781 conn(0x7fa4a4072960 msgr2=0x7fa4a413e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 shutdown_connections 2023-11-24T07:59:18.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.433+0000 7fa4a8c79700 1 -- 172.21.15.32:0/2987501781 wait complete. 2023-11-24T07:59:18.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.573+0000 7f6482dea700 1 Processor -- start 2023-11-24T07:59:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.573+0000 7f6482dea700 1 -- start start 2023-11-24T07:59:18.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6482dea700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f647c0aed00 0x7f647c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6482dea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c071a80 0x7f647c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6482dea700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c156910 con 0x7f647c0afa90 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6482dea700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c15c760 con 0x7f647c0aed00 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6482dea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c15ca40 con 0x7f647c071a80 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6480b86700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f647c0aed00 0x7f647c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6481387700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c071a80 0x7f647c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6480b86700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f647c0aed00 0x7f647c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:41972/0 (socket says 172.21.15.32:41972) 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f6480b86700 1 -- 172.21.15.32:0/2682430813 learned_addr learned my addr 172.21.15.32:0/2682430813 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.574+0000 7f647bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f647c0afa90 0x7f647c071370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:46290/0 (socket says 172.21.15.32:46290) 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f6480b86700 1 -- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c071a80 msgr2=0x7f647c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f6480b86700 1 --2- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c071a80 0x7f647c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f6480b86700 1 -- 172.21.15.32:0/2682430813 >> v1:172.21.15.123:6789/0 conn(0x7f647c0afa90 legacy=0x7f647c071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f6480b86700 1 -- 172.21.15.32:0/2682430813 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c15cd20 con 0x7f647c0aed00 2023-11-24T07:59:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f6480b86700 1 --2- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c0cfb10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64700098f0 tx=0x7f6470009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376884b980220641 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f647b7fe700 1 -- 172.21.15.32:0/2682430813 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470011830 con 0x7f647c0aed00 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f647b7fe700 1 -- 172.21.15.32:0/2682430813 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6470011990 con 0x7f647c0aed00 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.575+0000 7f647b7fe700 1 -- 172.21.15.32:0/2682430813 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470011d20 con 0x7f647c0aed00 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 -- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 msgr2=0x7f647c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 --2- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c0cfb10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64700098f0 tx=0x7f6470009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 -- 172.21.15.32:0/2682430813 shutdown_connections 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 --2- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c071a80 0x7f647c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 --2- 172.21.15.32:0/2682430813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c0cfb10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 -- 172.21.15.32:0/2682430813 >> 172.21.15.32:0/2682430813 conn(0x7f647c06aff0 msgr2=0x7f647c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 -- 172.21.15.32:0/2682430813 shutdown_connections 2023-11-24T07:59:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.576+0000 7f6482dea700 1 -- 172.21.15.32:0/2682430813 wait complete. 2023-11-24T07:59:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.577+0000 7f6482dea700 1 Processor -- start 2023-11-24T07:59:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.577+0000 7f6482dea700 1 -- start start 2023-11-24T07:59:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6482dea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 0x7f647c14a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6482dea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6482dea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c0afa90 0x7f647c1f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6482dea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c14f110 con 0x7f647c071a80 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6482dea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c14e630 con 0x7f647c0aed00 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6482dea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c14dd90 con 0x7f647c0afa90 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 0x7f647c14a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 0x7f647c14a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57828/0 (socket says 172.21.15.32:57828) 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 -- 172.21.15.32:0/3126594368 learned_addr learned my addr 172.21.15.32:0/3126594368 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f647bfff700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6481387700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c0afa90 0x7f647c1f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 -- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c0afa90 msgr2=0x7f647c1f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c0afa90 0x7f647c1f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 -- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 msgr2=0x7f647c1f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c1f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 -- 172.21.15.32:0/3126594368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470009550 con 0x7f647c071a80 2023-11-24T07:59:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.578+0000 7f6480b86700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 0x7f647c14a9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6470000c00 tx=0x7f6470004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.579+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64700036a0 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.579+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6470003800 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.579+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c14e070 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.579+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470003bb0 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.579+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c14b1c0 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.579+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c14f380 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.580+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6470021070 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.580+0000 7f64797fa700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f646006aed0 0x7f646006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.580+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f647008ab80 con 0x7f647c071a80 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.581+0000 7f647bfff700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f646006aed0 0x7f646006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.581+0000 7f647bfff700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f646006aed0 0x7f646006d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f647c15ed00 tx=0x7f6468007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.585+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6470055cc0 con 0x7f647c071a80 2023-11-24T07:59:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.787+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f647c06f1d0 con 0x7f647c071a80 2023-11-24T07:59:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.788+0000 7f64797fa700 1 -- 172.21.15.32:0/3126594368 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6470015020 con 0x7f647c071a80 2023-11-24T07:59:18.789 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f646006aed0 msgr2=0x7f646006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f646006aed0 0x7f646006d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f647c15ed00 tx=0x7f6468007ed0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 msgr2=0x7f647c14a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 0x7f647c14a9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6470000c00 tx=0x7f6470004750 comp rx=0 tx=0).stop 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 shutdown_connections 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f646006aed0 0x7f646006d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f647c0afa90 0x7f647c1f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f647c071a80 0x7f647c14a9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 --2- 172.21.15.32:0/3126594368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f647c0aed00 0x7f647c1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 >> 172.21.15.32:0/3126594368 conn(0x7f647c06aff0 msgr2=0x7f647c15e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 shutdown_connections 2023-11-24T07:59:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.790+0000 7f6482dea700 1 -- 172.21.15.32:0/3126594368 wait complete. 2023-11-24T07:59:18.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.928+0000 7f29ad97d700 1 Processor -- start 2023-11-24T07:59:18.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.928+0000 7f29ad97d700 1 -- start start 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29ad97d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29a815c140 0x7f29a80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29ad97d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80cc780 con 0x7f29a80cf2b0 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29ad97d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80cc9b0 con 0x7f29a815bd60 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29ad97d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80ccc90 con 0x7f29a815c140 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29a77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29a80cf2b0 0x7f29a80c57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59552/0 (socket says 172.21.15.32:59552) 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29a77fe700 1 -- 172.21.15.32:0/3551856896 learned_addr learned my addr 172.21.15.32:0/3551856896 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:18.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.929+0000 7f29a67fc700 1 --2- 172.21.15.32:0/3551856896 >> v2:172.21.15.123:3300/0 conn(0x7f29a815c140 0x7f29a80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a5ffb700 1 -- 172.21.15.32:0/3551856896 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182185417 0 0) 0x7f29a80cc780 con 0x7f29a80cf2b0 2023-11-24T07:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a5ffb700 1 -- 172.21.15.32:0/3551856896 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29940035b0 con 0x7f29a80cf2b0 2023-11-24T07:59:18.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a5ffb700 1 -- 172.21.15.32:0/3551856896 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824735171 0 0) 0x7f29940035b0 con 0x7f29a80cf2b0 2023-11-24T07:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a67fc700 1 -- 172.21.15.32:0/3551856896 >> v1:172.21.15.123:6789/0 conn(0x7f29a815bd60 legacy=0x7f29a80d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a67fc700 1 -- 172.21.15.32:0/3551856896 >> v1:172.21.15.32:6789/0 conn(0x7f29a80cf2b0 legacy=0x7f29a80c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a67fc700 1 -- 172.21.15.32:0/3551856896 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80ccf70 con 0x7f29a815c140 2023-11-24T07:59:18.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a67fc700 1 --2- 172.21.15.32:0/3551856896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a815c140 0x7f29a80cda50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2990009840 tx=0x7f2990009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661112e83da0b23 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a5ffb700 1 -- 172.21.15.32:0/3551856896 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998001670 con 0x7f29a815c140 2023-11-24T07:59:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a5ffb700 1 -- 172.21.15.32:0/3551856896 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990004030 con 0x7f29a815c140 2023-11-24T07:59:18.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.930+0000 7f29a5ffb700 1 -- 172.21.15.32:0/3551856896 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29900043a0 con 0x7f29a815c140 2023-11-24T07:59:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 -- 172.21.15.32:0/3551856896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a815c140 msgr2=0x7f29a80cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3551856896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a815c140 0x7f29a80cda50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2990009840 tx=0x7f2990009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 -- 172.21.15.32:0/3551856896 shutdown_connections 2023-11-24T07:59:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3551856896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a815c140 0x7f29a80cda50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 -- 172.21.15.32:0/3551856896 >> 172.21.15.32:0/3551856896 conn(0x7f29a8073130 msgr2=0x7f29a809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 -- 172.21.15.32:0/3551856896 shutdown_connections 2023-11-24T07:59:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.931+0000 7f29ad97d700 1 -- 172.21.15.32:0/3551856896 wait complete. 2023-11-24T07:59:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.932+0000 7f29ad97d700 1 Processor -- start 2023-11-24T07:59:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.932+0000 7f29ad97d700 1 -- start start 2023-11-24T07:59:18.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29ad97d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a80cf2b0 0x7f29a81f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29ad97d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 0x7f29a8156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29ad97d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29a815c140 0x7f29a81585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29ad97d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81576f0 con 0x7f29a815bd60 2023-11-24T07:59:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29ad97d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8156ae0 con 0x7f29a80cf2b0 2023-11-24T07:59:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29ad97d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8156dc0 con 0x7f29a815c140 2023-11-24T07:59:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 0x7f29a8156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 0x7f29a8156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57844/0 (socket says 172.21.15.32:57844) 2023-11-24T07:59:18.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 -- 172.21.15.32:0/3805503362 learned_addr learned my addr 172.21.15.32:0/3805503362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a6ffd700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a80cf2b0 0x7f29a81f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a77fe700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29a815c140 0x7f29a81585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 -- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29a815c140 msgr2=0x7f29a81585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29a815c140 0x7f29a81585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 -- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a80cf2b0 msgr2=0x7f29a81f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a80cf2b0 0x7f29a81f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a67fc700 1 -- 172.21.15.32:0/3805503362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29900094a0 con 0x7f29a815bd60 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.933+0000 7f29a77fe700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29a815c140 0x7f29a81585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.934+0000 7f29a67fc700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 0x7f29a8156170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f299000a010 tx=0x7f2990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.934+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990003e80 con 0x7f29a815bd60 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.934+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990010590 con 0x7f29a815bd60 2023-11-24T07:59:18.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.934+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a81570a0 con 0x7f29a815bd60 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.934+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990010900 con 0x7f29a815bd60 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.934+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a81f1a00 con 0x7f29a815bd60 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.935+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a81f1b90 con 0x7f29a815bd60 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.935+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2990010a60 con 0x7f29a815bd60 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.936+0000 7f299ffff700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f298806aed0 0x7f298806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.936+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f299008ae90 con 0x7f29a815bd60 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.936+0000 7f29a6ffd700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f298806aed0 0x7f298806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.936+0000 7f29a6ffd700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f298806aed0 0x7f298806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f29a0001ec0 tx=0x7f29a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:18.940+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2990055f50 con 0x7f29a815bd60 2023-11-24T07:59:19.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.153+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29a8067f60 con 0x7f29a815bd60 2023-11-24T07:59:19.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.154+0000 7f299ffff700 1 -- 172.21.15.32:0/3805503362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2990078070 con 0x7f29a815bd60 2023-11-24T07:59:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f298806aed0 msgr2=0x7f298806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f298806aed0 0x7f298806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f29a0001ec0 tx=0x7f29a0009040 comp rx=0 tx=0).stop 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 msgr2=0x7f29a8156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 0x7f29a8156170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f299000a010 tx=0x7f2990004750 comp rx=0 tx=0).stop 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 shutdown_connections 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f298806aed0 0x7f298806d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29a815c140 0x7f29a81585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a80cf2b0 0x7f29a81f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 --2- 172.21.15.32:0/3805503362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29a815bd60 0x7f29a8156170 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 >> 172.21.15.32:0/3805503362 conn(0x7f29a8073130 msgr2=0x7f29a80c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 shutdown_connections 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.156+0000 7f29ad97d700 1 -- 172.21.15.32:0/3805503362 wait complete. 2023-11-24T07:59:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-11-24T07:59:19.169 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-24T07:59:19.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:19.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-24T07:59:19.219 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:59:19.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:19.220 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T07:59:19.238 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T07:59:19.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:19.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.363+0000 7f17097e3700 1 Processor -- start 2023-11-24T07:59:19.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.364+0000 7f17097e3700 1 -- start start 2023-11-24T07:59:19.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17097e3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1704152cc0 0x7f170406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17097e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17040b49a0 0x7f170406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17097e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040cff20 0x7f17040d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17097e3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170415c4b0 con 0x7f17040b49a0 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17097e3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170415c790 con 0x7f1704152cc0 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17097e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170415ca70 con 0x7f17040cff20 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17040b49a0 0x7f170406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17040b49a0 0x7f170406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57852/0 (socket says 172.21.15.32:57852) 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 -- 172.21.15.32:0/2635524453 learned_addr learned my addr 172.21.15.32:0/2635524453 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f1702ffd700 1 --2- 172.21.15.32:0/2635524453 >> v2:172.21.15.123:3300/0 conn(0x7f1704152cc0 0x7f170406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17037fe700 1 --2- 172.21.15.32:0/2635524453 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040cff20 0x7f17040d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 -- 172.21.15.32:0/2635524453 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040cff20 msgr2=0x7f17040d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 --2- 172.21.15.32:0/2635524453 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040cff20 0x7f17040d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 -- 172.21.15.32:0/2635524453 >> v2:172.21.15.123:3300/0 conn(0x7f1704152cc0 msgr2=0x7f170406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 --2- 172.21.15.32:0/2635524453 >> v2:172.21.15.123:3300/0 conn(0x7f1704152cc0 0x7f170406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 -- 172.21.15.32:0/2635524453 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170415cd50 con 0x7f17040b49a0 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f1702ffd700 1 --2- 172.21.15.32:0/2635524453 >> v2:172.21.15.123:3300/0 conn(0x7f1704152cc0 0x7f170406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.365+0000 7f17027fc700 1 --2- 172.21.15.32:0/2635524453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040b49a0 0x7f170406c540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1704075030 tx=0x7f16ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c577a9a05067434a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f1701ffb700 1 -- 172.21.15.32:0/2635524453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00b870 con 0x7f17040b49a0 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f1701ffb700 1 -- 172.21.15.32:0/2635524453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec00b9d0 con 0x7f17040b49a0 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f1701ffb700 1 -- 172.21.15.32:0/2635524453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00bd60 con 0x7f17040b49a0 2023-11-24T07:59:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 -- 172.21.15.32:0/2635524453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040b49a0 msgr2=0x7f170406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 --2- 172.21.15.32:0/2635524453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040b49a0 0x7f170406c540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1704075030 tx=0x7f16ec009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 -- 172.21.15.32:0/2635524453 shutdown_connections 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 --2- 172.21.15.32:0/2635524453 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040cff20 0x7f17040d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 --2- 172.21.15.32:0/2635524453 >> v2:172.21.15.123:3300/0 conn(0x7f1704152cc0 0x7f170406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 --2- 172.21.15.32:0/2635524453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040b49a0 0x7f170406c540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 -- 172.21.15.32:0/2635524453 >> 172.21.15.32:0/2635524453 conn(0x7f1704071b90 msgr2=0x7f1704071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 -- 172.21.15.32:0/2635524453 shutdown_connections 2023-11-24T07:59:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.366+0000 7f17097e3700 1 -- 172.21.15.32:0/2635524453 wait complete. 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.367+0000 7f17097e3700 1 Processor -- start 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 -- start start 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040b49a0 0x7f17041f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 0x7f170406f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704152cc0 0x7f170415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17041f4030 con 0x7f17040cff20 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170406dec0 con 0x7f1704152cc0 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17097e3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170406e1a0 con 0x7f17040b49a0 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 0x7f170406f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 0x7f170406f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57858/0 (socket says 172.21.15.32:57858) 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.368+0000 7f17027fc700 1 -- 172.21.15.32:0/3671357888 learned_addr learned my addr 172.21.15.32:0/3671357888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17037fe700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704152cc0 0x7f170415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f1702ffd700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040b49a0 0x7f17041f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17027fc700 1 -- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040b49a0 msgr2=0x7f17041f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17027fc700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040b49a0 0x7f17041f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17027fc700 1 -- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704152cc0 msgr2=0x7f170415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17027fc700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704152cc0 0x7f170415e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17027fc700 1 -- 172.21.15.32:0/3671357888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec009550 con 0x7f17040cff20 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17027fc700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 0x7f170406f300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec004540 con 0x7f17040cff20 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec0036a0 con 0x7f17040cff20 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170406e480 con 0x7f17040cff20 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec003a50 con 0x7f17040cff20 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.369+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17041f7b40 con 0x7f17040cff20 2023-11-24T07:59:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.370+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17040753f0 con 0x7f17040cff20 2023-11-24T07:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.371+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16ec021070 con 0x7f17040cff20 2023-11-24T07:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.371+0000 7f16fb7fe700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f16e406aed0 0x7f16e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.371+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16ec08acb0 con 0x7f17040cff20 2023-11-24T07:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.371+0000 7f1702ffd700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f16e406aed0 0x7f16e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.372+0000 7f1702ffd700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f16e406aed0 0x7f16e406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f16f400c900 tx=0x7f16f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.375+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16ec055df0 con 0x7f17040cff20 2023-11-24T07:59:19.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.622+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17041f7cd0 con 0x7f17040cff20 2023-11-24T07:59:19.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.623+0000 7f16fb7fe700 1 -- 172.21.15.32:0/3671357888 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f16ec015020 con 0x7f17040cff20 2023-11-24T07:59:19.625 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:19.625 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:59:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.625+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f16e406aed0 msgr2=0x7f16e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.625+0000 7f17097e3700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f16e406aed0 0x7f16e406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f16f400c900 tx=0x7f16f4008040 comp rx=0 tx=0).stop 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.625+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 msgr2=0x7f170406f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.625+0000 7f17097e3700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 0x7f170406f300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec004750 comp rx=0 tx=0).stop 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 shutdown_connections 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f16e406aed0 0x7f16e406d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1704152cc0 0x7f170415e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17040b49a0 0x7f17041f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 --2- 172.21.15.32:0/3671357888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17040cff20 0x7f170406f300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 >> 172.21.15.32:0/3671357888 conn(0x7f1704071b90 msgr2=0x7f170406c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 shutdown_connections 2023-11-24T07:59:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.626+0000 7f17097e3700 1 -- 172.21.15.32:0/3671357888 wait complete. 2023-11-24T07:59:19.629 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:59:19.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.763+0000 7ff38cc89700 1 Processor -- start 2023-11-24T07:59:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.763+0000 7ff38cc89700 1 -- start start 2023-11-24T07:59:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff38cc89700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff38815e1e0 0x7ff3880d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff38cc89700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff3880d0f80 0x7ff3880b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff38cc89700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3880d20b0 con 0x7ff3880d3a00 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff38cc89700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3880d3150 con 0x7ff38815e1e0 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff38cc89700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38815e5c0 con 0x7ff3880d0f80 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff385d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3880d3a00 0x7ff3880d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59578/0 (socket says 172.21.15.32:59578) 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff385d9b700 1 -- 172.21.15.32:0/426546072 learned_addr learned my addr 172.21.15.32:0/426546072 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff38659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff38815e1e0 0x7ff3880d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.764+0000 7ff386d9d700 1 --2- 172.21.15.32:0/426546072 >> v2:172.21.15.123:3300/0 conn(0x7ff3880d0f80 0x7ff3880b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38559a700 1 -- 172.21.15.32:0/426546072 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754082800 0 0) 0x7ff3880d20b0 con 0x7ff3880d3a00 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38559a700 1 -- 172.21.15.32:0/426546072 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3700035b0 con 0x7ff3880d3a00 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38659c700 1 -- 172.21.15.32:0/426546072 >> v2:172.21.15.123:3300/0 conn(0x7ff3880d0f80 msgr2=0x7ff3880b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38659c700 1 --2- 172.21.15.32:0/426546072 >> v2:172.21.15.123:3300/0 conn(0x7ff3880d0f80 0x7ff3880b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38659c700 1 -- 172.21.15.32:0/426546072 >> v1:172.21.15.32:6789/0 conn(0x7ff3880d3a00 legacy=0x7ff3880d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38659c700 1 -- 172.21.15.32:0/426546072 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880d2390 con 0x7ff38815e1e0 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38659c700 1 --2- 172.21.15.32:0/426546072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38815e1e0 0x7ff3880d34c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff38808c640 tx=0x7ff37c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783c2d770caaccf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38559a700 1 -- 172.21.15.32:0/426546072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c011830 con 0x7ff38815e1e0 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38559a700 1 -- 172.21.15.32:0/426546072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c011990 con 0x7ff38815e1e0 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.765+0000 7ff38559a700 1 -- 172.21.15.32:0/426546072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c011d20 con 0x7ff38815e1e0 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 -- 172.21.15.32:0/426546072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38815e1e0 msgr2=0x7ff3880d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 --2- 172.21.15.32:0/426546072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38815e1e0 0x7ff3880d34c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff38808c640 tx=0x7ff37c009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 -- 172.21.15.32:0/426546072 shutdown_connections 2023-11-24T07:59:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 --2- 172.21.15.32:0/426546072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38815e1e0 0x7ff3880d34c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 --2- 172.21.15.32:0/426546072 >> v2:172.21.15.123:3300/0 conn(0x7ff3880d0f80 0x7ff3880b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 -- 172.21.15.32:0/426546072 >> 172.21.15.32:0/426546072 conn(0x7ff3880b5b50 msgr2=0x7ff3880b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 -- 172.21.15.32:0/426546072 shutdown_connections 2023-11-24T07:59:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.766+0000 7ff38cc89700 1 -- 172.21.15.32:0/426546072 wait complete. 2023-11-24T07:59:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.767+0000 7ff38cc89700 1 Processor -- start 2023-11-24T07:59:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.767+0000 7ff38cc89700 1 -- start start 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38cc89700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 0x7ff3880ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38cc89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3880d3a00 0x7ff38815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38cc89700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff38815e1e0 0x7ff3880cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38cc89700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38815f520 con 0x7ff3880d0f80 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38cc89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff388161220 con 0x7ff3880d3a00 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38cc89700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff388160c60 con 0x7ff38815e1e0 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 0x7ff3880ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 0x7ff3880ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57870/0 (socket says 172.21.15.32:57870) 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 -- 172.21.15.32:0/1354963201 learned_addr learned my addr 172.21.15.32:0/1354963201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff386d9d700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff38815e1e0 0x7ff3880cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff385d9b700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3880d3a00 0x7ff38815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 -- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff38815e1e0 msgr2=0x7ff3880cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff38815e1e0 0x7ff3880cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 -- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3880d3a00 msgr2=0x7ff38815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3880d3a00 0x7ff38815f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 -- 172.21.15.32:0/1354963201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0094a0 con 0x7ff3880d0f80 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.768+0000 7ff38659c700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 0x7ff3880ca870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0097c0 tx=0x7ff37c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.769+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0036a0 con 0x7ff3880d0f80 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.769+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c01e070 con 0x7ff3880d0f80 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.769+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff388160f40 con 0x7ff3880d0f80 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.769+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c003a60 con 0x7ff3880d0f80 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.769+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38815fef0 con 0x7ff3880d0f80 2023-11-24T07:59:19.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.770+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3881600b0 con 0x7ff3880d0f80 2023-11-24T07:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.770+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff37c003bc0 con 0x7ff3880d0f80 2023-11-24T07:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.770+0000 7ff376ffd700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ff36806aed0 0x7ff36806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.771+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff37c08a420 con 0x7ff3880d0f80 2023-11-24T07:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.771+0000 7ff385d9b700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ff36806aed0 0x7ff36806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.771+0000 7ff385d9b700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ff36806aed0 0x7ff36806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff388154f60 tx=0x7ff378009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.775+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff37c055560 con 0x7ff3880d0f80 2023-11-24T07:59:19.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.977+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3880ce730 con 0x7ff3880d0f80 2023-11-24T07:59:19.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.978+0000 7ff376ffd700 1 -- 172.21.15.32:0/1354963201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7ff37c058b80 con 0x7ff3880d0f80 2023-11-24T07:59:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:19.980 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ff36806aed0 msgr2=0x7ff36806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ff36806aed0 0x7ff36806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff388154f60 tx=0x7ff378009040 comp rx=0 tx=0).stop 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 msgr2=0x7ff3880ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 0x7ff3880ca870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0097c0 tx=0x7ff37c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 shutdown_connections 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7ff36806aed0 0x7ff36806d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff38815e1e0 0x7ff3880cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3880d0f80 0x7ff3880ca870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 --2- 172.21.15.32:0/1354963201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3880d3a00 0x7ff38815f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 >> 172.21.15.32:0/1354963201 conn(0x7ff3880b5b50 msgr2=0x7ff38815c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 shutdown_connections 2023-11-24T07:59:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:19.980+0000 7ff38cc89700 1 -- 172.21.15.32:0/1354963201 wait complete. 2023-11-24T07:59:19.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:35.663110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:37.044002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:19.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.121+0000 7f06ab934700 1 Processor -- start 2023-11-24T07:59:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.121+0000 7f06ab934700 1 -- start start 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06ab934700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06a415e160 0x7f06a40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06ab934700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a40b3c60 con 0x7f06a40afc90 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06ab934700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a40b58c0 con 0x7f06a40746a0 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06ab934700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40b4500 con 0x7f06a415e160 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a96d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06a415e160 0x7f06a40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a8ecf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f06a40afc90 0x7f06a4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59592/0 (socket says 172.21.15.32:59592) 2023-11-24T07:59:20.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a8ecf700 1 -- 172.21.15.32:0/3932908717 learned_addr learned my addr 172.21.15.32:0/3932908717 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f0697fff700 1 -- 172.21.15.32:0/3932908717 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669089646 0 0) 0x7f06a40b3c60 con 0x7f06a40afc90 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f0697fff700 1 -- 172.21.15.32:0/3932908717 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06900035b0 con 0x7f06a40afc90 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a96d0700 1 -- 172.21.15.32:0/3932908717 >> v1:172.21.15.123:6789/0 conn(0x7f06a40746a0 legacy=0x7f06a406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a96d0700 1 -- 172.21.15.32:0/3932908717 >> v1:172.21.15.32:6789/0 conn(0x7f06a40afc90 legacy=0x7f06a4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a96d0700 1 -- 172.21.15.32:0/3932908717 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40b0140 con 0x7f06a415e160 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f06a96d0700 1 --2- 172.21.15.32:0/3932908717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a415e160 0x7f06a40b5da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009840 tx=0x7f06a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65aba64439e68db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f0697fff700 1 -- 172.21.15.32:0/3932908717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011830 con 0x7f06a415e160 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f0697fff700 1 -- 172.21.15.32:0/3932908717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0011990 con 0x7f06a415e160 2023-11-24T07:59:20.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.122+0000 7f0697fff700 1 -- 172.21.15.32:0/3932908717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011d20 con 0x7f06a415e160 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 -- 172.21.15.32:0/3932908717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a415e160 msgr2=0x7f06a40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 --2- 172.21.15.32:0/3932908717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a415e160 0x7f06a40b5da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009840 tx=0x7f06a0009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 -- 172.21.15.32:0/3932908717 shutdown_connections 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 --2- 172.21.15.32:0/3932908717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a415e160 0x7f06a40b5da0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 -- 172.21.15.32:0/3932908717 >> 172.21.15.32:0/3932908717 conn(0x7f06a40d0bf0 msgr2=0x7f06a40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 -- 172.21.15.32:0/3932908717 shutdown_connections 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.123+0000 7f06ab934700 1 -- 172.21.15.32:0/3932908717 wait complete. 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.124+0000 7f06ab934700 1 Processor -- start 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.124+0000 7f06ab934700 1 -- start start 2023-11-24T07:59:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06ab934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 0x7f06a40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06ab934700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a40afc90 0x7f06a406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06ab934700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06a415e160 0x7f06a40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06ab934700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c65b0 con 0x7f06a40746a0 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06ab934700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c6890 con 0x7f06a40afc90 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06ab934700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c6b70 con 0x7f06a415e160 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a96d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 0x7f06a40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a96d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 0x7f06a40c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57884/0 (socket says 172.21.15.32:57884) 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a96d0700 1 -- 172.21.15.32:0/12077004 learned_addr learned my addr 172.21.15.32:0/12077004 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a8ecf700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a40afc90 0x7f06a406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a9ed1700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06a415e160 0x7f06a40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a96d0700 1 -- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06a415e160 msgr2=0x7f06a40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a96d0700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06a415e160 0x7f06a40c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.125+0000 7f06a96d0700 1 -- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a40afc90 msgr2=0x7f06a406eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f06a96d0700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a40afc90 0x7f06a406eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f06a96d0700 1 -- 172.21.15.32:0/12077004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00094a0 con 0x7f06a40746a0 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f06a8ecf700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a40afc90 0x7f06a406eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f06a96d0700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 0x7f06a40c7650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009b90 tx=0x7f06a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a00038a0 con 0x7f06a40746a0 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a001e070 con 0x7f06a40746a0 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a40c6e50 con 0x7f06a40746a0 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0003c40 con 0x7f06a40746a0 2023-11-24T07:59:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.126+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a406f0b0 con 0x7f06a40746a0 2023-11-24T07:59:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.127+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a4068fb0 con 0x7f06a40746a0 2023-11-24T07:59:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.127+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06a002d450 con 0x7f06a40746a0 2023-11-24T07:59:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.128+0000 7f0695ffb700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f068806aed0 0x7f068806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.128+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a008a570 con 0x7f06a40746a0 2023-11-24T07:59:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.128+0000 7f06a8ecf700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f068806aed0 0x7f068806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.129+0000 7f06a8ecf700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f068806aed0 0x7f068806d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06a4159170 tx=0x7f0698009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.132+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06a00556b0 con 0x7f06a40746a0 2023-11-24T07:59:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.330+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06a406c140 con 0x7f06a40746a0 2023-11-24T07:59:20.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.330+0000 7f0695ffb700 1 -- 172.21.15.32:0/12077004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06a0058cd0 con 0x7f06a40746a0 2023-11-24T07:59:20.332 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:20.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f068806aed0 msgr2=0x7f068806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f068806aed0 0x7f068806d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06a4159170 tx=0x7f0698009040 comp rx=0 tx=0).stop 2023-11-24T07:59:20.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 msgr2=0x7f06a40c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 0x7f06a40c7650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009b90 tx=0x7f06a0004750 comp rx=0 tx=0).stop 2023-11-24T07:59:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 shutdown_connections 2023-11-24T07:59:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f068806aed0 0x7f068806d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06a415e160 0x7f06a40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a40746a0 0x7f06a40c7650 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 --2- 172.21.15.32:0/12077004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06a40afc90 0x7f06a406eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 >> 172.21.15.32:0/12077004 conn(0x7f06a40d0bf0 msgr2=0x7f06a40ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 shutdown_connections 2023-11-24T07:59:20.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.332+0000 7f06ab934700 1 -- 172.21.15.32:0/12077004 wait complete. 2023-11-24T07:59:20.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 Processor -- start 2023-11-24T07:59:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 -- start start 2023-11-24T07:59:20.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0ef815bf50 0x7f0ef80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ef80ce830 0x7f0ef80cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80cf1f0 0x7f0ef80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef815c330 con 0x7f0ef80ce830 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef80d0140 con 0x7f0ef815bf50 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.473+0000 7f0efe5f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80d03e0 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ef80ce830 0x7f0ef80cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ef80ce830 0x7f0ef80cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57894/0 (socket says 172.21.15.32:57894) 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 -- 172.21.15.32:0/1009008701 learned_addr learned my addr 172.21.15.32:0/1009008701 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef7fff700 1 --2- 172.21.15.32:0/1009008701 >> v2:172.21.15.123:3300/0 conn(0x7f0ef815bf50 0x7f0ef80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0efcb94700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80cf1f0 0x7f0ef80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 -- 172.21.15.32:0/1009008701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80cf1f0 msgr2=0x7f0ef80c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80cf1f0 0x7f0ef80c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 -- 172.21.15.32:0/1009008701 >> v2:172.21.15.123:3300/0 conn(0x7f0ef815bf50 msgr2=0x7f0ef80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 --2- 172.21.15.32:0/1009008701 >> v2:172.21.15.123:3300/0 conn(0x7f0ef815bf50 0x7f0ef80ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 -- 172.21.15.32:0/1009008701 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef80d06c0 con 0x7f0ef80ce830 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0efcb94700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80cf1f0 0x7f0ef80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef7fff700 1 --2- 172.21.15.32:0/1009008701 >> v2:172.21.15.123:3300/0 conn(0x7f0ef815bf50 0x7f0ef80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef77fe700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80ce830 0x7f0ef80cecb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80715e0 tx=0x7f0eec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f34fd5a42fad68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef6ffd700 1 -- 172.21.15.32:0/1009008701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00b870 con 0x7f0ef80ce830 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef6ffd700 1 -- 172.21.15.32:0/1009008701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec00b9d0 con 0x7f0ef80ce830 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.474+0000 7f0ef6ffd700 1 -- 172.21.15.32:0/1009008701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00bd60 con 0x7f0ef80ce830 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 -- 172.21.15.32:0/1009008701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80ce830 msgr2=0x7f0ef80cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80ce830 0x7f0ef80cecb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80715e0 tx=0x7f0eec009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 -- 172.21.15.32:0/1009008701 shutdown_connections 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80cf1f0 0x7f0ef80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/1009008701 >> v2:172.21.15.123:3300/0 conn(0x7f0ef815bf50 0x7f0ef80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/1009008701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80ce830 0x7f0ef80cecb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 -- 172.21.15.32:0/1009008701 >> 172.21.15.32:0/1009008701 conn(0x7f0ef80b3780 msgr2=0x7f0ef80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 -- 172.21.15.32:0/1009008701 shutdown_connections 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.475+0000 7f0efe5f7700 1 -- 172.21.15.32:0/1009008701 wait complete. 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.476+0000 7f0efe5f7700 1 Processor -- start 2023-11-24T07:59:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.476+0000 7f0efe5f7700 1 -- start start 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efe5f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80ce830 0x7f0ef8159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efe5f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 0x7f0ef80c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efe5f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef815bf50 0x7f0ef80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efe5f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef815a0a0 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efe5f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef81f1ab0 con 0x7f0ef815bf50 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efe5f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef81f1d90 con 0x7f0ef80ce830 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 0x7f0ef80c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 0x7f0ef80c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57900/0 (socket says 172.21.15.32:57900) 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 -- 172.21.15.32:0/2325864409 learned_addr learned my addr 172.21.15.32:0/2325864409 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:20.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efcb94700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef815bf50 0x7f0ef80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef7fff700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80ce830 0x7f0ef8159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 -- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80ce830 msgr2=0x7f0ef8159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80ce830 0x7f0ef8159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 -- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef815bf50 msgr2=0x7f0ef80c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef815bf50 0x7f0ef80c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef77fe700 1 -- 172.21.15.32:0/2325864409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec009550 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0efcb94700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef815bf50 0x7f0ef80c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.477+0000 7f0ef7fff700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80ce830 0x7f0ef8159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.478+0000 7f0ef77fe700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 0x7f0ef80c7830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80723f0 tx=0x7f0eec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.478+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec004310 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.478+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec01f070 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.478+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef81f2070 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.478+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0036a0 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.478+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef809bc00 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.479+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef81f2590 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.479+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0eec021030 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.480+0000 7f0ef4ff9700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0ee406aed0 0x7f0ee406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.480+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0eec08aa40 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.480+0000 7f0ef7fff700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0ee406aed0 0x7f0ee406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.480+0000 7f0ef7fff700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0ee406aed0 0x7f0ee406d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8008890 tx=0x7f0ee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.484+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ef81f2590 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.685+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ef81f2590 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.686+0000 7f0ef4ff9700 1 -- 172.21.15.32:0/2325864409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ef81f2590 con 0x7f0ef80cf1f0 2023-11-24T07:59:20.688 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:20.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0ee406aed0 msgr2=0x7f0ee406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0ee406aed0 0x7f0ee406d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8008890 tx=0x7f0ee8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 msgr2=0x7f0ef80c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 0x7f0ef80c7830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80723f0 tx=0x7f0eec004750 comp rx=0 tx=0).stop 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 shutdown_connections 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f0ee406aed0 0x7f0ee406d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ef815bf50 0x7f0ef80c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ef80ce830 0x7f0ef8159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 --2- 172.21.15.32:0/2325864409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ef80cf1f0 0x7f0ef80c7830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 >> 172.21.15.32:0/2325864409 conn(0x7f0ef80b3780 msgr2=0x7f0ef806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 shutdown_connections 2023-11-24T07:59:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.688+0000 7f0efe5f7700 1 -- 172.21.15.32:0/2325864409 wait complete. 2023-11-24T07:59:20.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.827+0000 7fd1feb9a700 1 Processor -- start 2023-11-24T07:59:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.827+0000 7fd1feb9a700 1 -- start start 2023-11-24T07:59:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1feb9a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd1f80b4020 0x7fd1f8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1feb9a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f815bf70 con 0x7fd1f80744d0 2023-11-24T07:59:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1feb9a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f815ee00 con 0x7fd1f80af520 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1feb9a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f8074960 con 0x7fd1f80b4020 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1fd137700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1f80744d0 0x7fd1f806d910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59622/0 (socket says 172.21.15.32:59622) 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1fd137700 1 -- 172.21.15.32:0/3845286225 learned_addr learned my addr 172.21.15.32:0/3845286225 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1f7fff700 1 --2- 172.21.15.32:0/3845286225 >> v2:172.21.15.123:3300/0 conn(0x7fd1f80b4020 0x7fd1f8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608597610 0 0) 0x7fd1f0001490 con 0x7fd1f80744d0 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.828+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1e80035b0 con 0x7fd1f80744d0 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1420772522 0 0) 0x7fd1e80035b0 con 0x7fd1f80744d0 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 >> v2:172.21.15.123:3300/0 conn(0x7fd1f80b4020 msgr2=0x7fd1f8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 --2- 172.21.15.32:0/3845286225 >> v2:172.21.15.123:3300/0 conn(0x7fd1f80b4020 0x7fd1f8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 >> v1:172.21.15.123:6789/0 conn(0x7fd1f80af520 legacy=0x7fd1f80afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80b0240 con 0x7fd1f80744d0 2023-11-24T07:59:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fd1f0001b80 con 0x7fd1f80744d0 2023-11-24T07:59:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 >> v1:172.21.15.32:6789/0 conn(0x7fd1f80744d0 legacy=0x7fd1f806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1e8003a00 0x7fd1e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1e8004850 0x7fd1e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8005210 0x7fd1e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8074960 con 0x7fd1e8003a00 2023-11-24T07:59:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e800c1b0 con 0x7fd1e8005210 2023-11-24T07:59:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e800c370 con 0x7fd1e8004850 2023-11-24T07:59:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1e8003a00 0x7fd1e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fd137700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8005210 0x7fd1e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1f7fff700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1e8004850 0x7fd1e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 -- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1e8004850 msgr2=0x7fd1e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1e8004850 0x7fd1e8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 -- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8005210 msgr2=0x7fd1e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8005210 0x7fd1e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fd137700 1 -- 172.21.15.32:0/3845286225 reap_dead start 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 -- 172.21.15.32:0/3845286225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80035b0 con 0x7fd1e8003a00 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.829+0000 7fd1fc936700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1e8003a00 0x7fd1e8003e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0017c0 tx=0x7fd1e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ff3cb665d9f7e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0052d0 con 0x7fd1e8003a00 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec005450 con 0x7fd1e8003a00 2023-11-24T07:59:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1f77fe700 1 -- 172.21.15.32:0/3845286225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0038c0 con 0x7fd1e8003a00 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 -- 172.21.15.32:0/3845286225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1e8003a00 msgr2=0x7fd1e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1e8003a00 0x7fd1e8003e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0017c0 tx=0x7fd1e8005880 comp rx=0 tx=0).stop 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 -- 172.21.15.32:0/3845286225 shutdown_connections 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/3845286225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1e8003a00 0x7fd1e8003e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 -- 172.21.15.32:0/3845286225 >> 172.21.15.32:0/3845286225 conn(0x7fd1f80b28f0 msgr2=0x7fd1f80b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 -- 172.21.15.32:0/3845286225 shutdown_connections 2023-11-24T07:59:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.830+0000 7fd1feb9a700 1 -- 172.21.15.32:0/3845286225 wait complete. 2023-11-24T07:59:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.831+0000 7fd1feb9a700 1 Processor -- start 2023-11-24T07:59:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.831+0000 7fd1feb9a700 1 -- start start 2023-11-24T07:59:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1feb9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8003a00 0x7fd1f8155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1feb9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 0x7fd1f815f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1feb9a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1f814fd00 0x7fd1f815fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1feb9a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8155b10 con 0x7fd1f81564e0 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1feb9a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8158770 con 0x7fd1e8003a00 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1feb9a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f81554c0 con 0x7fd1f814fd00 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 0x7fd1f815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.832+0000 7fd1f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 0x7fd1f815f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57920/0 (socket says 172.21.15.32:57920) 2023-11-24T07:59:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 -- 172.21.15.32:0/908274536 learned_addr learned my addr 172.21.15.32:0/908274536 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1fc936700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8003a00 0x7fd1f8155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1fd137700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1f814fd00 0x7fd1f815fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 -- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1f814fd00 msgr2=0x7fd1f815fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1f814fd00 0x7fd1f815fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 -- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8003a00 msgr2=0x7fd1f8155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8003a00 0x7fd1f8155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 -- 172.21.15.32:0/908274536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f81557a0 con 0x7fd1f81564e0 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f7fff700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 0x7fd1f815f530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006dd0 tx=0x7fd1e4011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004030 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4004190 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f81551b0 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004540 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.833+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f815d170 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.834+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f8155df0 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.835+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1e400f940 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.835+0000 7fd1f57fa700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd1dc06aed0 0x7fd1dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.835+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1e4089970 con 0x7fd1f81564e0 2023-11-24T07:59:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.835+0000 7fd1fc936700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd1dc06aed0 0x7fd1dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.836+0000 7fd1fc936700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd1dc06aed0 0x7fd1dc06d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000c00 tx=0x7fd1ec009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:20.840+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1e4054a20 con 0x7fd1f81564e0 2023-11-24T07:59:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.039+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd1f8068fb0 con 0x7fd1f81564e0 2023-11-24T07:59:21.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.040+0000 7fd1f57fa700 1 -- 172.21.15.32:0/908274536 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1e4022020 con 0x7fd1f81564e0 2023-11-24T07:59:21.042 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd1dc06aed0 msgr2=0x7fd1dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd1dc06aed0 0x7fd1dc06d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000c00 tx=0x7fd1ec009030 comp rx=0 tx=0).stop 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 msgr2=0x7fd1f815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 0x7fd1f815f530 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006dd0 tx=0x7fd1e4011e00 comp rx=0 tx=0).stop 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 shutdown_connections 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd1dc06aed0 0x7fd1dc06d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1f814fd00 0x7fd1f815fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e8003a00 0x7fd1f8155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 --2- 172.21.15.32:0/908274536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f81564e0 0x7fd1f815f530 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 >> 172.21.15.32:0/908274536 conn(0x7fd1f80b28f0 msgr2=0x7fd1f80b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 shutdown_connections 2023-11-24T07:59:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.042+0000 7fd1feb9a700 1 -- 172.21.15.32:0/908274536 wait complete. 2023-11-24T07:59:21.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.179+0000 7f6974252700 1 Processor -- start 2023-11-24T07:59:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.179+0000 7f6974252700 1 -- start start 2023-11-24T07:59:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f6974252700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f696c15e1e0 0x7f696c0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f6974252700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f696c0d0f80 0x7f696c0b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f6974252700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696c0d20b0 con 0x7f696c0d3a00 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f6974252700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696c0d3150 con 0x7f696c0d0f80 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f6974252700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696c15e5c0 con 0x7f696c15e1e0 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f69727ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f696c0d0f80 0x7f696c0b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f69727ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f696c0d0f80 0x7f696c0b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57930/0 (socket says 172.21.15.32:57930) 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.180+0000 7f69727ef700 1 -- 172.21.15.32:0/657162875 learned_addr learned my addr 172.21.15.32:0/657162875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f6971fee700 1 --2- 172.21.15.32:0/657162875 >> v2:172.21.15.123:3300/0 conn(0x7f696c15e1e0 0x7f696c0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f69727ef700 1 -- 172.21.15.32:0/657162875 >> v2:172.21.15.123:3300/0 conn(0x7f696c15e1e0 msgr2=0x7f696c0d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f69727ef700 1 --2- 172.21.15.32:0/657162875 >> v2:172.21.15.123:3300/0 conn(0x7f696c15e1e0 0x7f696c0d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f69727ef700 1 -- 172.21.15.32:0/657162875 >> v1:172.21.15.123:6789/0 conn(0x7f696c0d3a00 legacy=0x7f696c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f69727ef700 1 -- 172.21.15.32:0/657162875 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0d2390 con 0x7f696c0d0f80 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f69727ef700 1 --2- 172.21.15.32:0/657162875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c0b6ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f696c0d0840 tx=0x7f696400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2315e6f4598445d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f6970fec700 1 -- 172.21.15.32:0/657162875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696400e070 con 0x7f696c0d0f80 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f6970fec700 1 -- 172.21.15.32:0/657162875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964003a20 con 0x7f696c0d0f80 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f6970fec700 1 -- 172.21.15.32:0/657162875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964003dd0 con 0x7f696c0d0f80 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.181+0000 7f6974252700 1 -- 172.21.15.32:0/657162875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 msgr2=0x7f696c0b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 --2- 172.21.15.32:0/657162875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c0b6ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f696c0d0840 tx=0x7f696400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 -- 172.21.15.32:0/657162875 shutdown_connections 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 --2- 172.21.15.32:0/657162875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c0b6ef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 --2- 172.21.15.32:0/657162875 >> v2:172.21.15.123:3300/0 conn(0x7f696c15e1e0 0x7f696c0d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 -- 172.21.15.32:0/657162875 >> 172.21.15.32:0/657162875 conn(0x7f696c0b5b50 msgr2=0x7f696c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 -- 172.21.15.32:0/657162875 shutdown_connections 2023-11-24T07:59:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.182+0000 7f6974252700 1 -- 172.21.15.32:0/657162875 wait complete. 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.183+0000 7f6974252700 1 Processor -- start 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.183+0000 7f6974252700 1 -- start start 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f696c0d3a00 0x7f696c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f696c15e1e0 0x7f696c15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15e700 con 0x7f696c0d0f80 2023-11-24T07:59:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15e950 con 0x7f696c0d3a00 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c15ec30 con 0x7f696c15e1e0 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57932/0 (socket says 172.21.15.32:57932) 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 -- 172.21.15.32:0/736449724 learned_addr learned my addr 172.21.15.32:0/736449724 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f69717ed700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f696c0d3a00 0x7f696c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f69727ef700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f696c15e1e0 0x7f696c15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 -- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f696c15e1e0 msgr2=0x7f696c15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f696c15e1e0 0x7f696c15fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 -- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f696c0d3a00 msgr2=0x7f696c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f696c0d3a00 0x7f696c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 -- 172.21.15.32:0/736449724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964009b30 con 0x7f696c0d0f80 2023-11-24T07:59:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6971fee700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c158500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f695c00d800 tx=0x7f695c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0125f0 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c012750 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c012b00 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c15ef10 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.184+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c161170 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.185+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c1613f0 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.186+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f695c012c60 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.186+0000 7f69627fc700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f695006aed0 0x7f695006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.186+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f695c0895b0 con 0x7f696c0d0f80 2023-11-24T07:59:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.187+0000 7f69717ed700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f695006aed0 0x7f695006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.187+0000 7f69717ed700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f695006aed0 0x7f695006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6968007fd0 tx=0x7f6968007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.191+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f695c0546f0 con 0x7f696c0d0f80 2023-11-24T07:59:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.434+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f696c15f430 con 0x7f696c0d0f80 2023-11-24T07:59:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.435+0000 7f69627fc700 1 -- 172.21.15.32:0/736449724 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f695c057d10 con 0x7f696c0d0f80 2023-11-24T07:59:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:21.437 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:59:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f695006aed0 msgr2=0x7f695006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f695006aed0 0x7f695006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6968007fd0 tx=0x7f6968007ec0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 msgr2=0x7f696c158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c158500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f695c00d800 tx=0x7f695c00db10 comp rx=0 tx=0).stop 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 shutdown_connections 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f695006aed0 0x7f695006d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f696c15e1e0 0x7f696c15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0d0f80 0x7f696c158500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 --2- 172.21.15.32:0/736449724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f696c0d3a00 0x7f696c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 >> 172.21.15.32:0/736449724 conn(0x7f696c0b5b50 msgr2=0x7f696c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 shutdown_connections 2023-11-24T07:59:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.437+0000 7f6974252700 1 -- 172.21.15.32:0/736449724 wait complete. 2023-11-24T07:59:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:59:21.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.575+0000 7fd0912a2700 1 Processor -- start 2023-11-24T07:59:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.575+0000 7fd0912a2700 1 -- start start 2023-11-24T07:59:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd0912a2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd08c15bdd0 0x7fd08c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd0912a2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd08c0b3370 0x7fd08c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd0912a2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c0b2030 con 0x7fd08c0d1c70 2023-11-24T07:59:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd0912a2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c0b2f50 con 0x7fd08c0b3370 2023-11-24T07:59:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd0912a2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c0b22d0 con 0x7fd08c15bdd0 2023-11-24T07:59:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd08a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd08c0b3370 0x7fd08c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd08a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd08c0b3370 0x7fd08c0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57944/0 (socket says 172.21.15.32:57944) 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd08a7fc700 1 -- 172.21.15.32:0/4218225488 learned_addr learned my addr 172.21.15.32:0/4218225488 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.576+0000 7fd08affd700 1 --2- 172.21.15.32:0/4218225488 >> v2:172.21.15.123:3300/0 conn(0x7fd08c15bdd0 0x7fd08c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd08a7fc700 1 -- 172.21.15.32:0/4218225488 >> v2:172.21.15.123:3300/0 conn(0x7fd08c15bdd0 msgr2=0x7fd08c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd08a7fc700 1 --2- 172.21.15.32:0/4218225488 >> v2:172.21.15.123:3300/0 conn(0x7fd08c15bdd0 0x7fd08c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd08a7fc700 1 -- 172.21.15.32:0/4218225488 >> v1:172.21.15.123:6789/0 conn(0x7fd08c0d1c70 legacy=0x7fd08c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd08a7fc700 1 -- 172.21.15.32:0/4218225488 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c0b25b0 con 0x7fd08c0b3370 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd08a7fc700 1 --2- 172.21.15.32:0/4218225488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c0b1af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd08c06ec10 tx=0x7fd074009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f512965fda566545 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd089ffb700 1 -- 172.21.15.32:0/4218225488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074011830 con 0x7fd08c0b3370 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd089ffb700 1 -- 172.21.15.32:0/4218225488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074011990 con 0x7fd08c0b3370 2023-11-24T07:59:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd089ffb700 1 -- 172.21.15.32:0/4218225488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074011d20 con 0x7fd08c0b3370 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd0912a2700 1 -- 172.21.15.32:0/4218225488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 msgr2=0x7fd08c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.577+0000 7fd0912a2700 1 --2- 172.21.15.32:0/4218225488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c0b1af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd08c06ec10 tx=0x7fd074009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.578+0000 7fd0912a2700 1 -- 172.21.15.32:0/4218225488 shutdown_connections 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.578+0000 7fd0912a2700 1 --2- 172.21.15.32:0/4218225488 >> v2:172.21.15.123:3300/0 conn(0x7fd08c15bdd0 0x7fd08c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.578+0000 7fd0912a2700 1 --2- 172.21.15.32:0/4218225488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c0b1af0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.578+0000 7fd0912a2700 1 -- 172.21.15.32:0/4218225488 >> 172.21.15.32:0/4218225488 conn(0x7fd08c0731a0 msgr2=0x7fd08c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.578+0000 7fd0912a2700 1 -- 172.21.15.32:0/4218225488 shutdown_connections 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.578+0000 7fd0912a2700 1 -- 172.21.15.32:0/4218225488 wait complete. 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.579+0000 7fd0912a2700 1 Processor -- start 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.579+0000 7fd0912a2700 1 -- start start 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0912a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0912a2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c0d1c70 0x7fd08c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0912a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c15bdd0 0x7fd08c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0912a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c06c4a0 con 0x7fd08c0b3370 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0912a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c06c780 con 0x7fd08c15bdd0 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0912a2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c06ca60 con 0x7fd08c0d1c70 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57946/0 (socket says 172.21.15.32:57946) 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 -- 172.21.15.32:0/3500725886 learned_addr learned my addr 172.21.15.32:0/3500725886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08b7fe700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c15bdd0 0x7fd08c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08a7fc700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c0d1c70 0x7fd08c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 -- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c0d1c70 msgr2=0x7fd08c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c0d1c70 0x7fd08c1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 -- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c15bdd0 msgr2=0x7fd08c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c15bdd0 0x7fd08c0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 -- 172.21.15.32:0/3500725886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074009550 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd08affd700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c15ecd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00e870 tx=0x7fd07c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.580+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c009590 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.581+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c013680 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.581+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0139f0 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.581+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c15f2b0 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.581+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c157610 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.581+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c06cdd0 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.582+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd07c013b50 con 0x7fd08c0b3370 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.582+0000 7fd0837fe700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06c06af00 0x7fd06c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.583+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd07c0893f0 con 0x7fd08c0b3370 2023-11-24T07:59:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.583+0000 7fd08a7fc700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06c06af00 0x7fd06c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.583+0000 7fd08a7fc700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06c06af00 0x7fd06c06d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b37e0 tx=0x7fd074008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.587+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd07c0544a0 con 0x7fd08c0b3370 2023-11-24T07:59:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.791+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd08c06c5e0 con 0x7fd08c0b3370 2023-11-24T07:59:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.792+0000 7fd0837fe700 1 -- 172.21.15.32:0/3500725886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fd08c06c5e0 con 0x7fd08c0b3370 2023-11-24T07:59:21.794 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:21.794 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06c06af00 msgr2=0x7fd06c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06c06af00 0x7fd06c06d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b37e0 tx=0x7fd074008030 comp rx=0 tx=0).stop 2023-11-24T07:59:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 msgr2=0x7fd08c15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c15ecd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00e870 tx=0x7fd07c00eb80 comp rx=0 tx=0).stop 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 shutdown_connections 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06c06af00 0x7fd06c06d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c15bdd0 0x7fd08c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd08c0b3370 0x7fd08c15ecd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 --2- 172.21.15.32:0/3500725886 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c0d1c70 0x7fd08c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.794+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 >> 172.21.15.32:0/3500725886 conn(0x7fd08c0731a0 msgr2=0x7fd08c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.795+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 shutdown_connections 2023-11-24T07:59:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.795+0000 7fd0912a2700 1 -- 172.21.15.32:0/3500725886 wait complete. 2023-11-24T07:59:21.810 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:59:21.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.933+0000 7f45413f9700 1 Processor -- start 2023-11-24T07:59:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.933+0000 7f45413f9700 1 -- start start 2023-11-24T07:59:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.934+0000 7f45413f9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 0x7f453c0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.934+0000 7f45413f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c15bb60 con 0x7f453c15ee90 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.934+0000 7f45413f9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c0b2bd0 con 0x7f453c0b0f80 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.934+0000 7f45413f9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b1fc0 con 0x7f453c0b0ba0 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.934+0000 7f453b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f453c15ee90 0x7f453c15b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59670/0 (socket says 172.21.15.32:59670) 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.934+0000 7f453b7fe700 1 -- 172.21.15.32:0/315891505 learned_addr learned my addr 172.21.15.32:0/315891505 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453affd700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 0x7f453c0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703925167 0 0) 0x7f4534001490 con 0x7f453c15ee90 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45240035b0 con 0x7f453c15ee90 2023-11-24T07:59:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2316610416 0 0) 0x7f45240035b0 con 0x7f453c15ee90 2023-11-24T07:59:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 msgr2=0x7f453c0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 0x7f453c0af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 >> v1:172.21.15.123:6789/0 conn(0x7f453c0b0f80 legacy=0x7f453c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c0b22a0 con 0x7f453c15ee90 2023-11-24T07:59:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f4534001b80 con 0x7f453c15ee90 2023-11-24T07:59:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 >> v1:172.21.15.32:6789/0 conn(0x7f453c15ee90 legacy=0x7f453c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 0x7f4524003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4524003f70 0x7f4524004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4524004d90 0x7f4524005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b1fc0 con 0x7f4524004d90 2023-11-24T07:59:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452400ba70 con 0x7f4524003f70 2023-11-24T07:59:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452400de40 con 0x7f453c0b0ba0 2023-11-24T07:59:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453b7fe700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4524004d90 0x7f4524005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453affd700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 0x7f4524003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453a7fc700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4524003f70 0x7f4524004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453b7fe700 1 -- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 msgr2=0x7f4524003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453b7fe700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c0b0ba0 0x7f4524003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453b7fe700 1 -- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4524003f70 msgr2=0x7f4524004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453b7fe700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4524003f70 0x7f4524004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.935+0000 7f453b7fe700 1 -- 172.21.15.32:0/315891505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240035b0 con 0x7f4524004d90 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f453b7fe700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4524004d90 0x7f4524005210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f453400a840 tx=0x7f4524005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81125827326bc036 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534012600 con 0x7f4524004d90 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534012760 con 0x7f4524004d90 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f4539ffb700 1 -- 172.21.15.32:0/315891505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534012af0 con 0x7f4524004d90 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f45413f9700 1 -- 172.21.15.32:0/315891505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4524004d90 msgr2=0x7f4524005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f45413f9700 1 --2- 172.21.15.32:0/315891505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4524004d90 0x7f4524005210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f453400a840 tx=0x7f4524005880 comp rx=0 tx=0).stop 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f453b7fe700 1 -- 172.21.15.32:0/315891505 reap_dead start 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f45413f9700 1 -- 172.21.15.32:0/315891505 shutdown_connections 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f45413f9700 1 -- 172.21.15.32:0/315891505 >> 172.21.15.32:0/315891505 conn(0x7f453c0b57b0 msgr2=0x7f453c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f45413f9700 1 -- 172.21.15.32:0/315891505 shutdown_connections 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.936+0000 7f45413f9700 1 -- 172.21.15.32:0/315891505 wait complete. 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 Processor -- start 2023-11-24T07:59:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 -- start start 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 0x7f453c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c151970 0x7f453c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453c0cc590 0x7f453c0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0aeb20 con 0x7f453c151590 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c09a490 con 0x7f453c0cc590 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f45413f9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0ae8e0 con 0x7f453c151970 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f453affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 0x7f453c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.938+0000 7f453affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 0x7f453c157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57966/0 (socket says 172.21.15.32:57966) 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 -- 172.21.15.32:0/4086636277 learned_addr learned my addr 172.21.15.32:0/4086636277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453b7fe700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453c0cc590 0x7f453c0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453a7fc700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c151970 0x7f453c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 -- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c151970 msgr2=0x7f453c0ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c151970 0x7f453c0ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 -- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453c0cc590 msgr2=0x7f453c0c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453c0cc590 0x7f453c0c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 -- 172.21.15.32:0/4086636277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c0cb7d0 con 0x7f453c151590 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f453affd700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 0x7f453c157750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4530011ad0 tx=0x7f4530011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000f8e0 con 0x7f453c151590 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000fa40 con 0x7f453c151590 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c0cbab0 con 0x7f453c151590 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000fdf0 con 0x7f453c151590 2023-11-24T07:59:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.939+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c0cc070 con 0x7f453c151590 2023-11-24T07:59:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.940+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c067f60 con 0x7f453c151590 2023-11-24T07:59:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.941+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4530014440 con 0x7f453c151590 2023-11-24T07:59:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.941+0000 7f45237fe700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f452806aed0 0x7f452806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.941+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4530089c30 con 0x7f453c151590 2023-11-24T07:59:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.941+0000 7f453a7fc700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f452806aed0 0x7f452806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.942+0000 7f453a7fc700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f452806aed0 0x7f452806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f452c007fd0 tx=0x7f452c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:21.945+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4530054d10 con 0x7f453c151590 2023-11-24T07:59:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.187+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f453c071f00 con 0x7f453c151590 2023-11-24T07:59:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.188+0000 7f45237fe700 1 -- 172.21.15.32:0/4086636277 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f4530058330 con 0x7f453c151590 2023-11-24T07:59:22.189 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:22.189 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:59:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f452806aed0 msgr2=0x7f452806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f452806aed0 0x7f452806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f452c007fd0 tx=0x7f452c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 msgr2=0x7f453c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 0x7f453c157750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4530011ad0 tx=0x7f4530011de0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 shutdown_connections 2023-11-24T07:59:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f452806aed0 0x7f452806d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f453c0cc590 0x7f453c0c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f453c151590 0x7f453c157750 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 --2- 172.21.15.32:0/4086636277 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f453c151970 0x7f453c0ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 >> 172.21.15.32:0/4086636277 conn(0x7f453c0b57b0 msgr2=0x7f453c0cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 shutdown_connections 2023-11-24T07:59:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.190+0000 7f45413f9700 1 -- 172.21.15.32:0/4086636277 wait complete. 2023-11-24T07:59:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:59:22.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:22.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.327+0000 7fd59aa1c700 1 Processor -- start 2023-11-24T07:59:22.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.327+0000 7fd59aa1c700 1 -- start start 2023-11-24T07:59:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd59aa1c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd59aa1c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5940b3370 0x7fd5940b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd59aa1c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5940d1c70 0x7fd5940d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd59aa1c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5940b2f50 con 0x7fd5940b3370 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd59aa1c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5940d2660 con 0x7fd5940d1c70 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd59aa1c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b2030 con 0x7fd59415bdd0 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd5937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5940b3370 0x7fd5940b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.328+0000 7fd5937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5940b3370 0x7fd5940b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57974/0 (socket says 172.21.15.32:57974) 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 -- 172.21.15.32:0/1189265919 learned_addr learned my addr 172.21.15.32:0/1189265919 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd598fb9700 1 --2- 172.21.15.32:0/1189265919 >> v2:172.21.15.123:3300/0 conn(0x7fd5940d1c70 0x7fd5940d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd593fff700 1 --2- 172.21.15.32:0/1189265919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 -- 172.21.15.32:0/1189265919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 msgr2=0x7fd5941604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 --2- 172.21.15.32:0/1189265919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 -- 172.21.15.32:0/1189265919 >> v2:172.21.15.123:3300/0 conn(0x7fd5940d1c70 msgr2=0x7fd5940d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 --2- 172.21.15.32:0/1189265919 >> v2:172.21.15.123:3300/0 conn(0x7fd5940d1c70 0x7fd5940d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 -- 172.21.15.32:0/1189265919 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940b22d0 con 0x7fd5940b3370 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd5937fe700 1 --2- 172.21.15.32:0/1189265919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940b3370 0x7fd5940b1af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd594160a20 tx=0x7fd588009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebda32e5d5222c8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd592ffd700 1 -- 172.21.15.32:0/1189265919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800b870 con 0x7fd5940b3370 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd592ffd700 1 -- 172.21.15.32:0/1189265919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58800b9d0 con 0x7fd5940b3370 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.329+0000 7fd592ffd700 1 -- 172.21.15.32:0/1189265919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800bd60 con 0x7fd5940b3370 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 -- 172.21.15.32:0/1189265919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940b3370 msgr2=0x7fd5940b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/1189265919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940b3370 0x7fd5940b1af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd594160a20 tx=0x7fd588009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 -- 172.21.15.32:0/1189265919 shutdown_connections 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/1189265919 >> v2:172.21.15.123:3300/0 conn(0x7fd5940d1c70 0x7fd5940d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/1189265919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/1189265919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940b3370 0x7fd5940b1af0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 -- 172.21.15.32:0/1189265919 >> 172.21.15.32:0/1189265919 conn(0x7fd5940731a0 msgr2=0x7fd59409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:22.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 -- 172.21.15.32:0/1189265919 shutdown_connections 2023-11-24T07:59:22.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.330+0000 7fd59aa1c700 1 -- 172.21.15.32:0/1189265919 wait complete. 2023-11-24T07:59:22.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.331+0000 7fd59aa1c700 1 Processor -- start 2023-11-24T07:59:22.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.331+0000 7fd59aa1c700 1 -- start start 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd59aa1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5940b3370 0x7fd5941e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd59aa1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 0x7fd5940c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd59aa1c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd59aa1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941e6780 con 0x7fd5940d1c70 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd59aa1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941e7420 con 0x7fd5940b3370 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd59aa1c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941e7700 con 0x7fd59415bdd0 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 0x7fd5940c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 0x7fd5940c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57980/0 (socket says 172.21.15.32:57980) 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 -- 172.21.15.32:0/838284519 learned_addr learned my addr 172.21.15.32:0/838284519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd593fff700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5940b3370 0x7fd5941e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd598fb9700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 -- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 msgr2=0x7fd5941e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941e6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 -- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5940b3370 msgr2=0x7fd5941e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5940b3370 0x7fd5941e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 -- 172.21.15.32:0/838284519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588009550 con 0x7fd5940d1c70 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd593fff700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5940b3370 0x7fd5941e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd598fb9700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.332+0000 7fd5937fe700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 0x7fd5940c7410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd588000c00 tx=0x7fd588004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.333+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588004520 con 0x7fd5940d1c70 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.333+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58801e070 con 0x7fd5940d1c70 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.333+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5880038c0 con 0x7fd5940d1c70 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.333+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5941e5760 con 0x7fd5940d1c70 2023-11-24T07:59:22.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.333+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5941e5d20 con 0x7fd5940d1c70 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.334+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5941e5eb0 con 0x7fd5940d1c70 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.334+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd588003a20 con 0x7fd5940d1c70 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.334+0000 7fd590ff9700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd58006aed0 0x7fd58006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.334+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd588020070 con 0x7fd5940d1c70 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.335+0000 7fd593fff700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd58006aed0 0x7fd58006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.335+0000 7fd593fff700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd58006aed0 0x7fd58006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd584009a00 tx=0x7fd584008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:22.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.339+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd588055c50 con 0x7fd5940d1c70 2023-11-24T07:59:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.583+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd594067f60 con 0x7fd5940d1c70 2023-11-24T07:59:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.584+0000 7fd590ff9700 1 -- 172.21.15.32:0/838284519 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fd58802a090 con 0x7fd5940d1c70 2023-11-24T07:59:22.586 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:22.586 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/440183025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":440183025},{"type":"v1","addr":"172.21.15.32:6835","nonce":440183025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/3162775518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":3162775518},{"type":"v1","addr":"172.21.15.32:6834","nonce":3162775518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1810149839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1810149839},{"type":"v1","addr":"172.21.15.123:6836","nonce":1810149839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:38.060811+0000","modified":"2023-11-24T07:58:41.071393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/1588575159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1588575159},{"type":"v1","addr":"172.21.15.123:6837","nonce":1588575159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:59:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd58006aed0 msgr2=0x7fd58006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd58006aed0 0x7fd58006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd584009a00 tx=0x7fd584008040 comp rx=0 tx=0).stop 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 msgr2=0x7fd5940c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 0x7fd5940c7410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd588000c00 tx=0x7fd588004750 comp rx=0 tx=0).stop 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 shutdown_connections 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd58006aed0 0x7fd58006d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59415bdd0 0x7fd5941e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5940b3370 0x7fd5941e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 --2- 172.21.15.32:0/838284519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5940d1c70 0x7fd5940c7410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 >> 172.21.15.32:0/838284519 conn(0x7fd5940731a0 msgr2=0x7fd59415a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 shutdown_connections 2023-11-24T07:59:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.586+0000 7fd59aa1c700 1 -- 172.21.15.32:0/838284519 wait complete. 2023-11-24T07:59:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-11-24T07:59:22.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.724+0000 7fe6158be700 1 Processor -- start 2023-11-24T07:59:22.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.724+0000 7fe6158be700 1 -- start start 2023-11-24T07:59:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe6158be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe610152cc0 0x7fe61006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe6158be700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100b49a0 0x7fe61006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe6158be700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe6100cff20 0x7fe6100d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe6158be700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61015c4b0 con 0x7fe610152cc0 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe6158be700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61015c790 con 0x7fe6100cff20 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe6158be700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61015ca70 con 0x7fe6100b49a0 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe60effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe610152cc0 0x7fe61006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe60effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe610152cc0 0x7fe61006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57986/0 (socket says 172.21.15.32:57986) 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.725+0000 7fe60effd700 1 -- 172.21.15.32:0/655301357 learned_addr learned my addr 172.21.15.32:0/655301357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60e7fc700 1 --2- 172.21.15.32:0/655301357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100b49a0 0x7fe61006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60f7fe700 1 --2- 172.21.15.32:0/655301357 >> v2:172.21.15.123:3300/0 conn(0x7fe6100cff20 0x7fe6100d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60effd700 1 -- 172.21.15.32:0/655301357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100b49a0 msgr2=0x7fe61006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60effd700 1 --2- 172.21.15.32:0/655301357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100b49a0 0x7fe61006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60effd700 1 -- 172.21.15.32:0/655301357 >> v2:172.21.15.123:3300/0 conn(0x7fe6100cff20 msgr2=0x7fe6100d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60effd700 1 --2- 172.21.15.32:0/655301357 >> v2:172.21.15.123:3300/0 conn(0x7fe6100cff20 0x7fe6100d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60effd700 1 -- 172.21.15.32:0/655301357 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61015cd50 con 0x7fe610152cc0 2023-11-24T07:59:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60f7fe700 1 --2- 172.21.15.32:0/655301357 >> v2:172.21.15.123:3300/0 conn(0x7fe6100cff20 0x7fe6100d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60effd700 1 --2- 172.21.15.32:0/655301357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe610152cc0 0x7fe61006c000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe610159e50 tx=0x7fe604009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32912a44e6c5604 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60dffb700 1 -- 172.21.15.32:0/655301357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604011830 con 0x7fe610152cc0 2023-11-24T07:59:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60dffb700 1 -- 172.21.15.32:0/655301357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604011990 con 0x7fe610152cc0 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.726+0000 7fe60dffb700 1 -- 172.21.15.32:0/655301357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604011d20 con 0x7fe610152cc0 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 -- 172.21.15.32:0/655301357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe610152cc0 msgr2=0x7fe61006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 --2- 172.21.15.32:0/655301357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe610152cc0 0x7fe61006c000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe610159e50 tx=0x7fe604009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 -- 172.21.15.32:0/655301357 shutdown_connections 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 --2- 172.21.15.32:0/655301357 >> v2:172.21.15.123:3300/0 conn(0x7fe6100cff20 0x7fe6100d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 --2- 172.21.15.32:0/655301357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe610152cc0 0x7fe61006c000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 --2- 172.21.15.32:0/655301357 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100b49a0 0x7fe61006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 -- 172.21.15.32:0/655301357 >> 172.21.15.32:0/655301357 conn(0x7fe610071b90 msgr2=0x7fe610071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 -- 172.21.15.32:0/655301357 shutdown_connections 2023-11-24T07:59:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.727+0000 7fe6158be700 1 -- 172.21.15.32:0/655301357 wait complete. 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.728+0000 7fe6158be700 1 Processor -- start 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.728+0000 7fe6158be700 1 -- start start 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe6158be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 0x7fe610147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe6158be700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100cff20 0x7fe610153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe6158be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe610152cc0 0x7fe61006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe6158be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610156920 con 0x7fe6100b49a0 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe6158be700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe610154650 con 0x7fe610152cc0 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe6158be700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe610154930 con 0x7fe6100cff20 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 0x7fe610147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 0x7fe610147910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57992/0 (socket says 172.21.15.32:57992) 2023-11-24T07:59:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 -- 172.21.15.32:0/3702286563 learned_addr learned my addr 172.21.15.32:0/3702286563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60e7fc700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100cff20 0x7fe610153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60f7fe700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe610152cc0 0x7fe61006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 -- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100cff20 msgr2=0x7fe610153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100cff20 0x7fe610153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 -- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe610152cc0 msgr2=0x7fe61006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe610152cc0 0x7fe61006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 -- 172.21.15.32:0/3702286563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604009550 con 0x7fe6100b49a0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.729+0000 7fe60effd700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 0x7fe610147910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe604009c10 tx=0x7fe604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.730+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040036a0 con 0x7fe6100b49a0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.730+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60401e070 con 0x7fe6100b49a0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.730+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61006e150 con 0x7fe6100b49a0 2023-11-24T07:59:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.730+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604003a40 con 0x7fe6100b49a0 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.730+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe610154c10 con 0x7fe6100b49a0 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.731+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6100753f0 con 0x7fe6100b49a0 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.731+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe604003ba0 con 0x7fe6100b49a0 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.732+0000 7fe5f7fff700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5fc06aed0 0x7fe5fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.732+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe60408a500 con 0x7fe6100b49a0 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.732+0000 7fe60e7fc700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5fc06aed0 0x7fe5fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.732+0000 7fe60e7fc700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5fc06aed0 0x7fe5fc06d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe610159260 tx=0x7fe600008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.736+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6040555c0 con 0x7fe6100b49a0 2023-11-24T07:59:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.939+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe61015f9e0 con 0x7fe6100b49a0 2023-11-24T07:59:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.940+0000 7fe5f7fff700 1 -- 172.21.15.32:0/3702286563 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fe61015f9e0 con 0x7fe6100b49a0 2023-11-24T07:59:22.942 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:22.942 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:58:39.648867+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:35.663110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:37.044002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5fc06aed0 msgr2=0x7fe5fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5fc06aed0 0x7fe5fc06d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe610159260 tx=0x7fe600008040 comp rx=0 tx=0).stop 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 msgr2=0x7fe610147910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 0x7fe610147910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe604009c10 tx=0x7fe604004750 comp rx=0 tx=0).stop 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 shutdown_connections 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe5fc06aed0 0x7fe5fc06d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe610152cc0 0x7fe61006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6100b49a0 0x7fe610147910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 --2- 172.21.15.32:0/3702286563 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6100cff20 0x7fe610153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 >> 172.21.15.32:0/3702286563 conn(0x7fe610071b90 msgr2=0x7fe61015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 shutdown_connections 2023-11-24T07:59:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:22.943+0000 7fe6158be700 1 -- 172.21.15.32:0/3702286563 wait complete. 2023-11-24T07:59:22.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:59:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.081+0000 7fcae545c700 1 Processor -- start 2023-11-24T07:59:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.081+0000 7fcae545c700 1 -- start start 2023-11-24T07:59:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcae545c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcae015dfc0 0x7fcae00d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcae545c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00d44e0 0x7fcae00d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcae545c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae00d2d10 con 0x7fcae00b2fc0 2023-11-24T07:59:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcae545c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae00cefd0 con 0x7fcae015dfc0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcae545c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00cf2b0 con 0x7fcae00d44e0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcade7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcae00b2fc0 0x7fcae0161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59712/0 (socket says 172.21.15.32:59712) 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcade7fc700 1 -- 172.21.15.32:0/1585209924 learned_addr learned my addr 172.21.15.32:0/1585209924 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcadf7fe700 1 --2- 172.21.15.32:0/1585209924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00d44e0 0x7fcae00d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.082+0000 7fcadeffd700 1 --2- 172.21.15.32:0/1585209924 >> v2:172.21.15.123:3300/0 conn(0x7fcae015dfc0 0x7fcae00d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcaddffb700 1 -- 172.21.15.32:0/1585209924 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755065703 0 0) 0x7fcae00d2d10 con 0x7fcae00b2fc0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcaddffb700 1 -- 172.21.15.32:0/1585209924 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcacc0035b0 con 0x7fcae00b2fc0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcadf7fe700 1 -- 172.21.15.32:0/1585209924 >> v2:172.21.15.123:3300/0 conn(0x7fcae015dfc0 msgr2=0x7fcae00d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcadf7fe700 1 --2- 172.21.15.32:0/1585209924 >> v2:172.21.15.123:3300/0 conn(0x7fcae015dfc0 0x7fcae00d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcadf7fe700 1 -- 172.21.15.32:0/1585209924 >> v1:172.21.15.32:6789/0 conn(0x7fcae00b2fc0 legacy=0x7fcae0161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcadf7fe700 1 -- 172.21.15.32:0/1585209924 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00d2fb0 con 0x7fcae00d44e0 2023-11-24T07:59:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcadeffd700 1 --2- 172.21.15.32:0/1585209924 >> v2:172.21.15.123:3300/0 conn(0x7fcae015dfc0 0x7fcae00d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcadf7fe700 1 --2- 172.21.15.32:0/1585209924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00d44e0 0x7fcae00d13a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009d90 tx=0x7fcad800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e758601918c33e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcaddffb700 1 -- 172.21.15.32:0/1585209924 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800e070 con 0x7fcae00d44e0 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcaddffb700 1 -- 172.21.15.32:0/1585209924 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad800bb20 con 0x7fcae00d44e0 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.083+0000 7fcaddffb700 1 -- 172.21.15.32:0/1585209924 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8012430 con 0x7fcae00d44e0 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 -- 172.21.15.32:0/1585209924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00d44e0 msgr2=0x7fcae00d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 --2- 172.21.15.32:0/1585209924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00d44e0 0x7fcae00d13a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009d90 tx=0x7fcad800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 -- 172.21.15.32:0/1585209924 shutdown_connections 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 --2- 172.21.15.32:0/1585209924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00d44e0 0x7fcae00d13a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 --2- 172.21.15.32:0/1585209924 >> v2:172.21.15.123:3300/0 conn(0x7fcae015dfc0 0x7fcae00d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 -- 172.21.15.32:0/1585209924 >> 172.21.15.32:0/1585209924 conn(0x7fcae006b410 msgr2=0x7fcae006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 -- 172.21.15.32:0/1585209924 shutdown_connections 2023-11-24T07:59:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.084+0000 7fcae545c700 1 -- 172.21.15.32:0/1585209924 wait complete. 2023-11-24T07:59:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.085+0000 7fcae545c700 1 Processor -- start 2023-11-24T07:59:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.085+0000 7fcae545c700 1 -- start start 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcae545c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b2fc0 0x7fcae0145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcae545c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 0x7fcae015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcae545c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015dfc0 0x7fcae015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcae545c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00c8260 con 0x7fcae00d44e0 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcae545c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0145790 con 0x7fcae015dfc0 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcae545c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01f3990 con 0x7fcae00b2fc0 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 0x7fcae015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 0x7fcae015f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58008/0 (socket says 172.21.15.32:58008) 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 -- 172.21.15.32:0/2573681984 learned_addr learned my addr 172.21.15.32:0/2573681984 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcadf7fe700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015dfc0 0x7fcae015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcadeffd700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b2fc0 0x7fcae0145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 -- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b2fc0 msgr2=0x7fcae0145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b2fc0 0x7fcae0145250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 -- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015dfc0 msgr2=0x7fcae015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015dfc0 0x7fcae015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 -- 172.21.15.32:0/2573681984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80094a0 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.086+0000 7fcade7fc700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 0x7fcae015f570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcad0001eb0 tx=0x7fcad000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.087+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0005d70 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.087+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad000ce70 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.087+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad8009a70 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.087+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00072d0 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.087+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae00c7ba0 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.088+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae00c83a0 con 0x7fcae00d44e0 2023-11-24T07:59:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.088+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcad0007430 con 0x7fcae00d44e0 2023-11-24T07:59:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.088+0000 7fcad77fe700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac006aed0 0x7fcac006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.089+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcad0088970 con 0x7fcae00d44e0 2023-11-24T07:59:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.089+0000 7fcadeffd700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac006aed0 0x7fcac006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.089+0000 7fcadeffd700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac006aed0 0x7fcac006d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d70 tx=0x7fcac8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.093+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcae00c83a0 con 0x7fcae00d44e0 2023-11-24T07:59:23.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.341+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcae00c83a0 con 0x7fcae00d44e0 2023-11-24T07:59:23.343 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-11-24T07:59:23.341+0000 7f61267f6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:59:23.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.342+0000 7fcad77fe700 1 -- 172.21.15.32:0/2573681984 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcae00c83a0 con 0x7fcae00d44e0 2023-11-24T07:59:23.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac006aed0 msgr2=0x7fcac006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac006aed0 0x7fcac006d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d70 tx=0x7fcac8009210 comp rx=0 tx=0).stop 2023-11-24T07:59:23.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 msgr2=0x7fcae015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 0x7fcae015f570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcad0001eb0 tx=0x7fcad000c7b0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 shutdown_connections 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcac006aed0 0x7fcac006d390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae015dfc0 0x7fcae015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcae00b2fc0 0x7fcae0145250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 --2- 172.21.15.32:0/2573681984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcae00d44e0 0x7fcae015f570 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 >> 172.21.15.32:0/2573681984 conn(0x7fcae006b410 msgr2=0x7fcae006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 shutdown_connections 2023-11-24T07:59:23.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.346+0000 7fcae545c700 1 -- 172.21.15.32:0/2573681984 wait complete. 2023-11-24T07:59:23.350 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:59:23.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:59:23.386 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-11-24T07:59:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.487+0000 7f97b3395700 1 Processor -- start 2023-11-24T07:59:23.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.487+0000 7f97b3395700 1 -- start start 2023-11-24T07:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b3395700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97ac0b29b0 0x7f97ac0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b3395700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac0c6180 0x7f97ac0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b3395700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac062960 con 0x7f97ac15ae60 2023-11-24T07:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b3395700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac062680 con 0x7f97ac0b29b0 2023-11-24T07:59:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b3395700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0cf250 con 0x7f97ac0c6180 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1131700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97ac15ae60 0x7f97ac0ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59726/0 (socket says 172.21.15.32:59726) 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1131700 1 -- 172.21.15.32:0/919047642 learned_addr learned my addr 172.21.15.32:0/919047642 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b0930700 1 --2- 172.21.15.32:0/919047642 >> v2:172.21.15.123:3300/0 conn(0x7f97ac0b29b0 0x7f97ac0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1932700 1 --2- 172.21.15.32:0/919047642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac0c6180 0x7f97ac0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97a3fff700 1 -- 172.21.15.32:0/919047642 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588517367 0 0) 0x7f97ac062960 con 0x7f97ac15ae60 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97a3fff700 1 -- 172.21.15.32:0/919047642 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97980035b0 con 0x7f97ac15ae60 2023-11-24T07:59:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1932700 1 -- 172.21.15.32:0/919047642 >> v2:172.21.15.123:3300/0 conn(0x7f97ac0b29b0 msgr2=0x7f97ac0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1932700 1 --2- 172.21.15.32:0/919047642 >> v2:172.21.15.123:3300/0 conn(0x7f97ac0b29b0 0x7f97ac0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1932700 1 -- 172.21.15.32:0/919047642 >> v1:172.21.15.32:6789/0 conn(0x7f97ac15ae60 legacy=0x7f97ac0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97a3fff700 1 -- 172.21.15.32:0/919047642 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2035746772 0 0) 0x7f97980035b0 con 0x7f97ac15ae60 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1932700 1 -- 172.21.15.32:0/919047642 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0cf530 con 0x7f97ac0c6180 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.488+0000 7f97b1932700 1 --2- 172.21.15.32:0/919047642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac0c6180 0x7f97ac0b0820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009d90 tx=0x7f97a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49acb7d5ae00b662 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97a3fff700 1 -- 172.21.15.32:0/919047642 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400e070 con 0x7f97ac0c6180 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97a3fff700 1 -- 172.21.15.32:0/919047642 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a4003a20 con 0x7f97ac0c6180 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97a3fff700 1 -- 172.21.15.32:0/919047642 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8001670 con 0x7f97ac0c6180 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 -- 172.21.15.32:0/919047642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac0c6180 msgr2=0x7f97ac0b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 --2- 172.21.15.32:0/919047642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac0c6180 0x7f97ac0b0820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009d90 tx=0x7f97a400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 -- 172.21.15.32:0/919047642 shutdown_connections 2023-11-24T07:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 --2- 172.21.15.32:0/919047642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac0c6180 0x7f97ac0b0820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 --2- 172.21.15.32:0/919047642 >> v2:172.21.15.123:3300/0 conn(0x7f97ac0b29b0 0x7f97ac0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 -- 172.21.15.32:0/919047642 >> 172.21.15.32:0/919047642 conn(0x7f97ac0b1e90 msgr2=0x7f97ac0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 -- 172.21.15.32:0/919047642 shutdown_connections 2023-11-24T07:59:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.489+0000 7f97b3395700 1 -- 172.21.15.32:0/919047642 wait complete. 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.491+0000 7f97b3395700 1 Processor -- start 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.491+0000 7f97b3395700 1 -- start start 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b3395700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97ac0b29b0 0x7f97ac14e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b3395700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 0x7f97ac15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b3395700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac15ae60 0x7f97ac155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b3395700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0af840 con 0x7f97ac0c6180 2023-11-24T07:59:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b3395700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ad210 con 0x7f97ac0b29b0 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b3395700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac15dcd0 con 0x7f97ac15ae60 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 0x7f97ac15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 0x7f97ac15b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58022/0 (socket says 172.21.15.32:58022) 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 -- 172.21.15.32:0/2292483993 learned_addr learned my addr 172.21.15.32:0/2292483993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b1131700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97ac0b29b0 0x7f97ac14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b1932700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac15ae60 0x7f97ac155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 -- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac15ae60 msgr2=0x7f97ac155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac15ae60 0x7f97ac155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 -- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97ac0b29b0 msgr2=0x7f97ac14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97ac0b29b0 0x7f97ac14e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 -- 172.21.15.32:0/2292483993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4009a70 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.492+0000 7f97b0930700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 0x7f97ac15b5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f979c00e9d0 tx=0x7f979c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.493+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c009730 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.493+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00c850 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.493+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac15dfb0 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.493+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00cbe0 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.493+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac0ad4b0 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.494+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0cfe60 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.494+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f979c00cd40 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.495+0000 7f97a1ffb700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f979006aed0 0x7f979006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.495+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f979c089990 con 0x7f97ac0c6180 2023-11-24T07:59:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.495+0000 7f97b1131700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f979006aed0 0x7f979006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.495+0000 7f97b1131700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f979006aed0 0x7f979006d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97a8001eb0 tx=0x7f97a8001c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.499+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f979c054830 con 0x7f97ac0c6180 2023-11-24T07:59:23.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:23.747+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f97ac0ad6d0 con 0x7f97ac0c6180 2023-11-24T07:59:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.343+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2292483993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f979c057e50 con 0x7f97ac0c6180 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f979006aed0 msgr2=0x7f979006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f979006aed0 0x7f979006d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97a8001eb0 tx=0x7f97a8001c40 comp rx=0 tx=0).stop 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 msgr2=0x7f97ac15b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 0x7f97ac15b5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f979c00e9d0 tx=0x7f979c00ed90 comp rx=0 tx=0).stop 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 shutdown_connections 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f979006aed0 0x7f979006d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97ac15ae60 0x7f97ac155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97ac0b29b0 0x7f97ac14e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 --2- 172.21.15.32:0/2292483993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac0c6180 0x7f97ac15b5c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 >> 172.21.15.32:0/2292483993 conn(0x7f97ac0b1e90 msgr2=0x7f97ac0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 shutdown_connections 2023-11-24T07:59:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.346+0000 7f97b3395700 1 -- 172.21.15.32:0/2292483993 wait complete. 2023-11-24T07:59:24.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:59:24.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.497+0000 7f5de5184700 1 Processor -- start 2023-11-24T07:59:24.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.497+0000 7f5de5184700 1 -- start start 2023-11-24T07:59:24.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5de5184700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5de00aefc0 0x7f5de00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5de5184700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5de00ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5de5184700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de00b0e50 con 0x7f5de00b19c0 2023-11-24T07:59:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5de5184700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de015ea90 con 0x7f5de00aefc0 2023-11-24T07:59:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5de5184700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00b02b0 con 0x7f5de00af3a0 2023-11-24T07:59:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5ddf59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5de00b19c0 0x7f5de015ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59744/0 (socket says 172.21.15.32:59744) 2023-11-24T07:59:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5ddf59e700 1 -- 172.21.15.32:0/88982302 learned_addr learned my addr 172.21.15.32:0/88982302 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5dded9d700 1 --2- 172.21.15.32:0/88982302 >> v2:172.21.15.123:3300/0 conn(0x7f5de00aefc0 0x7f5de00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5dde59c700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5de00ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626175710 0 0) 0x7f5de00b0e50 con 0x7f5de00b19c0 2023-11-24T07:59:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.498+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc80035b0 con 0x7f5de00b19c0 2023-11-24T07:59:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3617638053 0 0) 0x7f5dc80035b0 con 0x7f5de00b19c0 2023-11-24T07:59:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 msgr2=0x7f5de00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5de00ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 >> v2:172.21.15.123:3300/0 conn(0x7f5de00aefc0 msgr2=0x7f5de00ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 --2- 172.21.15.32:0/88982302 >> v2:172.21.15.123:3300/0 conn(0x7f5de00aefc0 0x7f5de00ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de00b0590 con 0x7f5de00b19c0 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f5dd80019f0 con 0x7f5de00b19c0 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dded9d700 1 --2- 172.21.15.32:0/88982302 >> v2:172.21.15.123:3300/0 conn(0x7f5de00aefc0 0x7f5de00ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dde59c700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5de00ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 >> v1:172.21.15.32:6789/0 conn(0x7f5de00b19c0 legacy=0x7f5de015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5dc8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc8003f40 0x7f5dc8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc8004d90 0x7f5dc8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de015ea90 con 0x7f5dc8004d90 2023-11-24T07:59:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00b02b0 con 0x7f5dc8003f40 2023-11-24T07:59:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc800ba10 con 0x7f5de00af3a0 2023-11-24T07:59:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc8004d90 0x7f5dc8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dde59c700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc8003f40 0x7f5dc8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5dded9d700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5dc8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 -- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 msgr2=0x7f5dc8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00af3a0 0x7f5dc8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 -- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc8003f40 msgr2=0x7f5dc8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc8003f40 0x7f5dc8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 -- 172.21.15.32:0/88982302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd00094a0 con 0x7f5dc8004d90 2023-11-24T07:59:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.499+0000 7f5ddf59e700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc8004d90 0x7f5dc8005210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800a830 tx=0x7f5dc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bde00cbe8955fad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012600 con 0x7f5dc8004d90 2023-11-24T07:59:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8012760 con 0x7f5dc8004d90 2023-11-24T07:59:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5dddd9b700 1 -- 172.21.15.32:0/88982302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012b10 con 0x7f5dc8004d90 2023-11-24T07:59:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5de5184700 1 -- 172.21.15.32:0/88982302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc8004d90 msgr2=0x7f5dc8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5de5184700 1 --2- 172.21.15.32:0/88982302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc8004d90 0x7f5dc8005210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800a830 tx=0x7f5dc8005880 comp rx=0 tx=0).stop 2023-11-24T07:59:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5ddf59e700 1 -- 172.21.15.32:0/88982302 reap_dead start 2023-11-24T07:59:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5de5184700 1 -- 172.21.15.32:0/88982302 shutdown_connections 2023-11-24T07:59:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5de5184700 1 -- 172.21.15.32:0/88982302 >> 172.21.15.32:0/88982302 conn(0x7f5de0072710 msgr2=0x7f5de0072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5de5184700 1 -- 172.21.15.32:0/88982302 shutdown_connections 2023-11-24T07:59:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.500+0000 7f5de5184700 1 -- 172.21.15.32:0/88982302 wait complete. 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 Processor -- start 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 -- start start 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 0x7f5de00c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de009ce50 0x7f5de015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00b55a0 0x7f5de015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de009d980 con 0x7f5de00b51c0 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de009d3c0 con 0x7f5de009ce50 2023-11-24T07:59:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5de5184700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de009d6a0 con 0x7f5de00b55a0 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.502+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 0x7f5de00c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 0x7f5de00c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58040/0 (socket says 172.21.15.32:58040) 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 -- 172.21.15.32:0/2612973122 learned_addr learned my addr 172.21.15.32:0/2612973122 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dde59c700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de009ce50 0x7f5de015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5ddf59e700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00b55a0 0x7f5de015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 -- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00b55a0 msgr2=0x7f5de015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00b55a0 0x7f5de015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 -- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de009ce50 msgr2=0x7f5de015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de009ce50 0x7f5de015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 -- 172.21.15.32:0/2612973122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de00c74c0 con 0x7f5de00b51c0 2023-11-24T07:59:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dde59c700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de009ce50 0x7f5de015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dded9d700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 0x7f5de00c65c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4011c20 tx=0x7f5dd400f240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400fb80 con 0x7f5de00b51c0 2023-11-24T07:59:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400fce0 con 0x7f5de00b51c0 2023-11-24T07:59:24.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de00c77a0 con 0x7f5de00b51c0 2023-11-24T07:59:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004240 con 0x7f5de00b51c0 2023-11-24T07:59:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.503+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de009b330 con 0x7f5de00b51c0 2023-11-24T07:59:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.504+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de0067f60 con 0x7f5de00b51c0 2023-11-24T07:59:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.505+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5dd40043a0 con 0x7f5de00b51c0 2023-11-24T07:59:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.505+0000 7f5dc77fe700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5dcc06aed0 0x7f5dcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.505+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5dd4089c50 con 0x7f5de00b51c0 2023-11-24T07:59:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.506+0000 7f5dde59c700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5dcc06aed0 0x7f5dcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.506+0000 7f5dde59c700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5dcc06aed0 0x7f5dcc06d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5de00af7b0 tx=0x7f5dd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.510+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5dd408f020 con 0x7f5de00b51c0 2023-11-24T07:59:24.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:24.713+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5de0073450 con 0x7f5de00b51c0 2023-11-24T07:59:25.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.644+0000 7f5dc77fe700 1 -- 172.21.15.32:0/2612973122 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5dd4022070 con 0x7f5de00b51c0 2023-11-24T07:59:25.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.647+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5dcc06aed0 msgr2=0x7f5dcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.647+0000 7f5de5184700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5dcc06aed0 0x7f5dcc06d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5de00af7b0 tx=0x7f5dd0008040 comp rx=0 tx=0).stop 2023-11-24T07:59:25.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.647+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 msgr2=0x7f5de00c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.647+0000 7f5de5184700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 0x7f5de00c65c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4011c20 tx=0x7f5dd400f240 comp rx=0 tx=0).stop 2023-11-24T07:59:25.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 shutdown_connections 2023-11-24T07:59:25.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f5dcc06aed0 0x7f5dcc06d390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de00b55a0 0x7f5de015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de00b51c0 0x7f5de00c65c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 --2- 172.21.15.32:0/2612973122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de009ce50 0x7f5de015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 >> 172.21.15.32:0/2612973122 conn(0x7f5de0072710 msgr2=0x7f5de015e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 shutdown_connections 2023-11-24T07:59:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.648+0000 7f5de5184700 1 -- 172.21.15.32:0/2612973122 wait complete. 2023-11-24T07:59:25.653 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:59:25.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:59:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.802+0000 7fef36e18700 1 Processor -- start 2023-11-24T07:59:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.802+0000 7fef36e18700 1 -- start start 2023-11-24T07:59:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef36e18700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef30144600 0x7fef300b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef36e18700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef3006f480 0x7fef3006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef36e18700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006e830 0x7fef3006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef36e18700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3006aff0 con 0x7fef3006f480 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef36e18700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3006f1c0 con 0x7fef30144600 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef36e18700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3006b200 con 0x7fef3006e830 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef3006f480 0x7fef3006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef3006f480 0x7fef3006c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58050/0 (socket says 172.21.15.32:58050) 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 -- 172.21.15.32:0/1998782643 learned_addr learned my addr 172.21.15.32:0/1998782643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef353b5700 1 --2- 172.21.15.32:0/1998782643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006e830 0x7fef3006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef34bb4700 1 --2- 172.21.15.32:0/1998782643 >> v2:172.21.15.123:3300/0 conn(0x7fef30144600 0x7fef300b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 -- 172.21.15.32:0/1998782643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006e830 msgr2=0x7fef3006ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 --2- 172.21.15.32:0/1998782643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006e830 0x7fef3006ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 -- 172.21.15.32:0/1998782643 >> v2:172.21.15.123:3300/0 conn(0x7fef30144600 msgr2=0x7fef300b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 --2- 172.21.15.32:0/1998782643 >> v2:172.21.15.123:3300/0 conn(0x7fef30144600 0x7fef300b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 -- 172.21.15.32:0/1998782643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3006b4e0 con 0x7fef3006f480 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef34bb4700 1 --2- 172.21.15.32:0/1998782643 >> v2:172.21.15.123:3300/0 conn(0x7fef30144600 0x7fef300b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.803+0000 7fef2ffff700 1 --2- 172.21.15.32:0/1998782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3006f480 0x7fef3006c2a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef300734a0 tx=0x7fef1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faff7b51c9f86f4d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef2f7fe700 1 -- 172.21.15.32:0/1998782643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00b870 con 0x7fef3006f480 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef2f7fe700 1 -- 172.21.15.32:0/1998782643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c00b9d0 con 0x7fef3006f480 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef2f7fe700 1 -- 172.21.15.32:0/1998782643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00bd60 con 0x7fef3006f480 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 -- 172.21.15.32:0/1998782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3006f480 msgr2=0x7fef3006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 --2- 172.21.15.32:0/1998782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3006f480 0x7fef3006c2a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef300734a0 tx=0x7fef1c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 -- 172.21.15.32:0/1998782643 shutdown_connections 2023-11-24T07:59:25.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 --2- 172.21.15.32:0/1998782643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006e830 0x7fef3006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 --2- 172.21.15.32:0/1998782643 >> v2:172.21.15.123:3300/0 conn(0x7fef30144600 0x7fef300b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 --2- 172.21.15.32:0/1998782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef3006f480 0x7fef3006c2a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 -- 172.21.15.32:0/1998782643 >> 172.21.15.32:0/1998782643 conn(0x7fef300d0b40 msgr2=0x7fef300d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 -- 172.21.15.32:0/1998782643 shutdown_connections 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.804+0000 7fef36e18700 1 -- 172.21.15.32:0/1998782643 wait complete. 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.805+0000 7fef36e18700 1 Processor -- start 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 -- start start 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef3006e830 0x7fef300aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006f480 0x7fef30158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 0x7fef30158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef300cf2e0 con 0x7fef30144600 2023-11-24T07:59:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3015cca0 con 0x7fef3006e830 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.806+0000 7fef36e18700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300ced20 con 0x7fef3006f480 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 0x7fef30158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 0x7fef30158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58058/0 (socket says 172.21.15.32:58058) 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 -- 172.21.15.32:0/2970787147 learned_addr learned my addr 172.21.15.32:0/2970787147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef2ffff700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006f480 0x7fef30158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef34bb4700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef3006e830 0x7fef300aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 -- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006f480 msgr2=0x7fef30158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006f480 0x7fef30158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 -- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef3006e830 msgr2=0x7fef300aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef3006e830 0x7fef300aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 -- 172.21.15.32:0/2970787147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c009550 con 0x7fef30144600 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef34bb4700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef3006e830 0x7fef300aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef353b5700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 0x7fef30158a60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef28007e60 tx=0x7fef2800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2800ede0 con 0x7fef30144600 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef28014420 con 0x7fef30144600 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef300cf000 con 0x7fef30144600 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280147d0 con 0x7fef30144600 2023-11-24T07:59:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.807+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3015d170 con 0x7fef30144600 2023-11-24T07:59:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.808+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3015d330 con 0x7fef30144600 2023-11-24T07:59:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.809+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef28014930 con 0x7fef30144600 2023-11-24T07:59:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.809+0000 7fef2d7fa700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fef1406aed0 0x7fef1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.809+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fef28089b10 con 0x7fef30144600 2023-11-24T07:59:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.809+0000 7fef34bb4700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fef1406aed0 0x7fef1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.810+0000 7fef34bb4700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fef1406aed0 0x7fef1406d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef30167dc0 tx=0x7fef24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:25.814+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef28054db0 con 0x7fef30144600 2023-11-24T07:59:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.016+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef300cfa20 con 0x7fef30144600 2023-11-24T07:59:26.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.649+0000 7fef2d7fa700 1 -- 172.21.15.32:0/2970787147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fef280583d0 con 0x7fef30144600 2023-11-24T07:59:26.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fef1406aed0 msgr2=0x7fef1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fef1406aed0 0x7fef1406d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef30167dc0 tx=0x7fef24008040 comp rx=0 tx=0).stop 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 msgr2=0x7fef30158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 0x7fef30158a60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef28007e60 tx=0x7fef2800e3c0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 shutdown_connections 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fef1406aed0 0x7fef1406d390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef30144600 0x7fef30158a60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef3006e830 0x7fef300aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 --2- 172.21.15.32:0/2970787147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef3006f480 0x7fef30158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.655+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 >> 172.21.15.32:0/2970787147 conn(0x7fef300d0b40 msgr2=0x7fef300d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.656+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 shutdown_connections 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.656+0000 7fef36e18700 1 -- 172.21.15.32:0/2970787147 wait complete. 2023-11-24T07:59:26.659 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-11-24T07:59:26.674 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:59:26.675 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T07:59:26.675 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T07:59:26.675 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:26.676 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:59:26.676 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T07:59:26.676 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T07:59:26.676 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:26.677 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:59:26.677 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T07:59:26.677 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T07:59:26.677 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:26.677 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:59:26.678 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T07:59:26.678 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T07:59:26.678 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:26.685 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T07:59:26.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:59:26.687 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T07:59:26.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:59:26.689 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T07:59:26.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:59:26.690 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T07:59:26.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:59:26.728 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:59:26.729 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:26.730 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:59:26.730 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:26.732 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:59:26.732 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:26.732 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:59:26.732 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:26.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T07:59:26.815 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-11-24T07:59:26.816 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-11-24T07:59:26.816 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-11-24T07:59:26.817 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-11-24T07:59:26.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.867+0000 7f94cf615700 1 Processor -- start 2023-11-24T07:59:26.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.867+0000 7f94cf615700 1 -- start start 2023-11-24T07:59:26.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cf615700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94c81446a0 0x7f94c80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cf615700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94c806c710 0x7f94c80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cf615700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c80af950 con 0x7f94c80adf00 2023-11-24T07:59:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cf615700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c8144a80 con 0x7f94c806c710 2023-11-24T07:59:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cf615700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c80afc30 con 0x7f94c81446a0 2023-11-24T07:59:26.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cdbb2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94c806c710 0x7f94c80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cdbb2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94c806c710 0x7f94c80b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58090/0 (socket says 172.21.15.32:58090) 2023-11-24T07:59:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cdbb2700 1 -- 172.21.15.32:0/3592624966 learned_addr learned my addr 172.21.15.32:0/3592624966 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.868+0000 7f94cd3b1700 1 --2- 172.21.15.32:0/3592624966 >> v2:172.21.15.123:3300/0 conn(0x7f94c81446a0 0x7f94c80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cdbb2700 1 -- 172.21.15.32:0/3592624966 >> v2:172.21.15.123:3300/0 conn(0x7f94c81446a0 msgr2=0x7f94c80b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cdbb2700 1 --2- 172.21.15.32:0/3592624966 >> v2:172.21.15.123:3300/0 conn(0x7f94c81446a0 0x7f94c80b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cdbb2700 1 -- 172.21.15.32:0/3592624966 >> v1:172.21.15.32:6789/0 conn(0x7f94c80adf00 legacy=0x7f94c806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cdbb2700 1 -- 172.21.15.32:0/3592624966 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c80aff10 con 0x7f94c806c710 2023-11-24T07:59:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cdbb2700 1 --2- 172.21.15.32:0/3592624966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c806c710 0x7f94c80b3d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f94c80729a0 tx=0x7f94c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13038b144f320695 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94bffff700 1 -- 172.21.15.32:0/3592624966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000e070 con 0x7f94c806c710 2023-11-24T07:59:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94bffff700 1 -- 172.21.15.32:0/3592624966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c000b9c0 con 0x7f94c806c710 2023-11-24T07:59:26.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94bffff700 1 -- 172.21.15.32:0/3592624966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000bd70 con 0x7f94c806c710 2023-11-24T07:59:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cf615700 1 -- 172.21.15.32:0/3592624966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c806c710 msgr2=0x7f94c80b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cf615700 1 --2- 172.21.15.32:0/3592624966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c806c710 0x7f94c80b3d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f94c80729a0 tx=0x7f94c000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cf615700 1 -- 172.21.15.32:0/3592624966 shutdown_connections 2023-11-24T07:59:26.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.869+0000 7f94cf615700 1 --2- 172.21.15.32:0/3592624966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c806c710 0x7f94c80b3d40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.870+0000 7f94cf615700 1 --2- 172.21.15.32:0/3592624966 >> v2:172.21.15.123:3300/0 conn(0x7f94c81446a0 0x7f94c80b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.870+0000 7f94cf615700 1 -- 172.21.15.32:0/3592624966 >> 172.21.15.32:0/3592624966 conn(0x7f94c8071b90 msgr2=0x7f94c8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:26.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.870+0000 7f94cf615700 1 -- 172.21.15.32:0/3592624966 shutdown_connections 2023-11-24T07:59:26.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.870+0000 7f94cf615700 1 -- 172.21.15.32:0/3592624966 wait complete. 2023-11-24T07:59:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.871+0000 7f94cf615700 1 Processor -- start 2023-11-24T07:59:26.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.871+0000 7f94cf615700 1 -- start start 2023-11-24T07:59:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94c806c710 0x7f94c815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94c80adf00 0x7f94c815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 0x7f94c8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81eaed0 con 0x7f94c81446a0 2023-11-24T07:59:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c815c390 con 0x7f94c80adf00 2023-11-24T07:59:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c815dea0 con 0x7f94c806c710 2023-11-24T07:59:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 0x7f94c8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 0x7f94c8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58096/0 (socket says 172.21.15.32:58096) 2023-11-24T07:59:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 -- 172.21.15.32:0/1966037853 learned_addr learned my addr 172.21.15.32:0/1966037853 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94ccbb0700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94c80adf00 0x7f94c815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cd3b1700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94c806c710 0x7f94c815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 -- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94c806c710 msgr2=0x7f94c815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94c806c710 0x7f94c815d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 -- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94c80adf00 msgr2=0x7f94c815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94c80adf00 0x7f94c815d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 -- 172.21.15.32:0/1966037853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0009b30 con 0x7f94c81446a0 2023-11-24T07:59:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cdbb2700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 0x7f94c8073f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f94c000b370 tx=0x7f94c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004900 con 0x7f94c81446a0 2023-11-24T07:59:26.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c00099b0 con 0x7f94c81446a0 2023-11-24T07:59:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c815e150 con 0x7f94c81446a0 2023-11-24T07:59:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.872+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c001c680 con 0x7f94c81446a0 2023-11-24T07:59:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.873+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c815c880 con 0x7f94c81446a0 2023-11-24T07:59:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.874+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c80753f0 con 0x7f94c81446a0 2023-11-24T07:59:26.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.874+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94c001c7e0 con 0x7f94c81446a0 2023-11-24T07:59:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.874+0000 7f94bdffb700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94ac06aed0 0x7f94ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.874+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f94c008aad0 con 0x7f94c81446a0 2023-11-24T07:59:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.875+0000 7f94cd3b1700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94ac06aed0 0x7f94ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.875+0000 7f94cd3b1700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94ac06aed0 0x7f94ac06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94b8006b20 tx=0x7f94b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:26.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:26.879+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94c0055f10 con 0x7f94c81446a0 2023-11-24T07:59:27.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.096+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f94c8067f60 con 0x7f94c81446a0 2023-11-24T07:59:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.660+0000 7f94bdffb700 1 -- 172.21.15.32:0/1966037853 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94c0059530 con 0x7f94c81446a0 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94ac06aed0 msgr2=0x7f94ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94ac06aed0 0x7f94ac06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94b8006b20 tx=0x7f94b8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 msgr2=0x7f94c8073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 0x7f94c8073f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f94c000b370 tx=0x7f94c0004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 shutdown_connections 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94ac06aed0 0x7f94ac06d390 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94b8006b20 tx=0x7f94b8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c81446a0 0x7f94c8073f90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94c806c710 0x7f94c815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 --2- 172.21.15.32:0/1966037853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94c80adf00 0x7f94c815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 >> 172.21.15.32:0/1966037853 conn(0x7f94c8071b90 msgr2=0x7f94c806df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:27.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.664+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 shutdown_connections 2023-11-24T07:59:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.666+0000 7f94cf615700 1 -- 172.21.15.32:0/1966037853 wait complete. 2023-11-24T07:59:27.669 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-11-24T07:59:27.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T07:59:27.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.815+0000 7fb28b746700 1 Processor -- start 2023-11-24T07:59:27.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.815+0000 7fb28b746700 1 -- start start 2023-11-24T07:59:27.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb28b746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 0x7fb2841604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb28b746700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840b22d0 con 0x7fb2840d1c70 2023-11-24T07:59:27.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb28b746700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840b25b0 con 0x7fb2840b3370 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb28b746700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840b2f50 con 0x7fb28415bdd0 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb289ce3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb2840d1c70 0x7fb284165ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59816/0 (socket says 172.21.15.32:59816) 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb289ce3700 1 -- 172.21.15.32:0/2380225505 learned_addr learned my addr 172.21.15.32:0/2380225505 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982698059 0 0) 0x7fb2840b22d0 con 0x7fb2840d1c70 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 0x7fb2841604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.816+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2700035b0 con 0x7fb2840d1c70 2023-11-24T07:59:27.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1087745082 0 0) 0x7fb2700035b0 con 0x7fb2840d1c70 2023-11-24T07:59:27.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 msgr2=0x7fb2841604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:27.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 0x7fb2841604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 >> v1:172.21.15.123:6789/0 conn(0x7fb2840b3370 legacy=0x7fb2840b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840b2890 con 0x7fb2840d1c70 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fb27c0019f0 con 0x7fb2840d1c70 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 >> v1:172.21.15.32:6789/0 conn(0x7fb2840d1c70 legacy=0x7fb284165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb270003a30 0x7fb270003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 0x7fb270004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb270004d90 0x7fb270005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840b2f50 con 0x7fb270003a30 2023-11-24T07:59:27.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27000bac0 con 0x7fb270004d90 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27000bc80 con 0x7fb28415bdd0 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb270003a30 0x7fb270003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb288ce1700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 0x7fb270004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb289ce3700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb270004d90 0x7fb270005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 -- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 msgr2=0x7fb270004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb28415bdd0 0x7fb270004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 -- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb270004d90 msgr2=0x7fb270005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb270004d90 0x7fb270005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 -- 172.21.15.32:0/2380225505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700035b0 con 0x7fb270003a30 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb270003a30 0x7fb270003e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb274009b30 tx=0x7fb270005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75c814a56e9484f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400e6c0 con 0x7fb270003a30 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27400e820 con 0x7fb270003a30 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.817+0000 7fb27bfff700 1 -- 172.21.15.32:0/2380225505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400eb90 con 0x7fb270003a30 2023-11-24T07:59:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb28b746700 1 -- 172.21.15.32:0/2380225505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb270003a30 msgr2=0x7fb270003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb28b746700 1 --2- 172.21.15.32:0/2380225505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb270003a30 0x7fb270003e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb274009b30 tx=0x7fb270005880 comp rx=0 tx=0).stop 2023-11-24T07:59:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb289ce3700 1 -- 172.21.15.32:0/2380225505 reap_dead start 2023-11-24T07:59:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb28b746700 1 -- 172.21.15.32:0/2380225505 shutdown_connections 2023-11-24T07:59:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb28b746700 1 -- 172.21.15.32:0/2380225505 >> 172.21.15.32:0/2380225505 conn(0x7fb2840731a0 msgr2=0x7fb28409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb28b746700 1 -- 172.21.15.32:0/2380225505 shutdown_connections 2023-11-24T07:59:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.818+0000 7fb28b746700 1 -- 172.21.15.32:0/2380225505 wait complete. 2023-11-24T07:59:27.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.819+0000 7fb28b746700 1 Processor -- start 2023-11-24T07:59:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.819+0000 7fb28b746700 1 -- start start 2023-11-24T07:59:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb28b746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb270003a30 0x7fb2840b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb28b746700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2840c6d40 0x7fb2840c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb28b746700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 0x7fb2840cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb28b746700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ce0a0 con 0x7fb2840cd6d0 2023-11-24T07:59:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb28b746700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284154c60 con 0x7fb2840c6d40 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb28b746700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840c7850 con 0x7fb270003a30 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb289ce3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 0x7fb2840cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb289ce3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 0x7fb2840cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58114/0 (socket says 172.21.15.32:58114) 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb289ce3700 1 -- 172.21.15.32:0/2343226854 learned_addr learned my addr 172.21.15.32:0/2343226854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb270003a30 0x7fb2840b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb288ce1700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2840c6d40 0x7fb2840c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.820+0000 7fb289ce3700 1 -- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb270003a30 msgr2=0x7fb2840b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb289ce3700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb270003a30 0x7fb2840b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb289ce3700 1 -- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2840c6d40 msgr2=0x7fb2840c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb289ce3700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2840c6d40 0x7fb2840c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb289ce3700 1 -- 172.21.15.32:0/2343226854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284154ed0 con 0x7fb2840cd6d0 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb288ce1700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2840c6d40 0x7fb2840c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb289ce3700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 0x7fb2840cdb60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2840b2200 tx=0x7fb27c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00adf0 con 0x7fb2840cd6d0 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c0133f0 con 0x7fb2840cd6d0 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2841551b0 con 0x7fb2840cd6d0 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0137a0 con 0x7fb2840cd6d0 2023-11-24T07:59:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.821+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb284159540 con 0x7fb2840cd6d0 2023-11-24T07:59:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.822+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284155800 con 0x7fb2840cd6d0 2023-11-24T07:59:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.822+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb27c013900 con 0x7fb2840cd6d0 2023-11-24T07:59:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.823+0000 7fb279ffb700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb26806aed0 0x7fb26806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.823+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb27c089c30 con 0x7fb2840cd6d0 2023-11-24T07:59:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.823+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb26806aed0 0x7fb26806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.824+0000 7fb2894e2700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb26806aed0 0x7fb26806d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2740090b0 tx=0x7fb27401f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:27.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:27.828+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb27c055030 con 0x7fb2840cd6d0 2023-11-24T07:59:28.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.109+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb284160480 con 0x7fb2840cd6d0 2023-11-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.109+0000 7fb279ffb700 1 -- 172.21.15.32:0/2343226854 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fb27c058650 con 0x7fb2840cd6d0 2023-11-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-11-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.111 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB3V2Bl7qkFKhAAWFInAhjEdGK+2n1KAh7cjA==", 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-11-24T07:59:28.112 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB3V2Bl7AkPNxAAXAxKAeK7bMmmbpkFcrmiyQ==", 2023-11-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-11-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-11-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.113 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-11-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB3V2BluWtuMRAALOz2bKOCcqTnzdzCHyTDow==", 2023-11-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-11-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-11-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-11-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB4V2BlF/gTAxAAiEFfMvHjFG1sdfEWlhG6eA==", 2023-11-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-11-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-11-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-11-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB5V2BlVwDPJBAAvXfR6CbEhbJvgvppkmhI9w==", 2023-11-24T07:59:28.118 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-11-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB8V2BljdIFJRAARdb3oYMz0SPcwqWAvIJgBA==", 2023-11-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-11-24T07:59:28.121 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB/V2BlCYR4JhAAKfqAy+WiJUp3Hkfluzow/w==", 2023-11-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-11-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCCV2BloCZPJhAA9qI9L0H2XR0yaTZ9Pwq62Q==", 2023-11-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-11-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCIV2Bl2g+eLhAAxJQVO6RZg05jW5ZWyO9S7A==", 2023-11-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-11-24T07:59:28.126 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCLV2BlMmG9LhAAtPFiB98UittUzKj9vCx73g==", 2023-11-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-11-24T07:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCOV2BlgeybLhAAQmviFD27q3NMO6ld08IBAA==", 2023-11-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCRV2BlALMULhAAKTKladOU8UiR0vHK9iwAVg==", 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB4V2Bl4D6UBxAADXMYHy5S2dPfRBlHDFLvHg==", 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.131 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB4V2BluMN/DhAAMHfkACUnwXufuREwxUli8Q==", 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-11-24T07:59:28.132 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB4V2BlrJAXCxAAOHrSLph/HvP0oOtOiSlTVQ==", 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB4V2BlinzJERAAz0p2a6j4pjcOAI7F6uSxqg==", 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB3V2BljVrtCBAAnRK53gfkKBe/iVc9XZ49LQ==", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-11-24T07:59:28.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCcV2BlftUGMhAAofcO/glEhSt+w6P6sonD/A==", 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCcV2BlSuEGMhAALwyTMO2qLZX02+jfI7VGMg==", 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCcV2BlkeoGMhAAobEl74DlhqCaBCMrP6kfbg==", 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T07:59:28.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCcV2BlzfcGMhAA86Q1bg0/bp5KOeB3Udg5HQ==", 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCcV2BlsAAHMhAAxr3GpMHh6zYsleunIDCmVw==", 2023-11-24T07:59:28.137 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQCcV2Bl1gkHMhAAER1PuqJX3kf1OICI+gjVaA==", 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB3V2BlpwCEJhAA13REsSfAK0sL34XS00P0Wg==", 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQB3V2BleKYeHxAA+MBP3PkXhXtzpqBgQjA6Iw==", 2023-11-24T07:59:28.139 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T07:59:28.140 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb26806aed0 msgr2=0x7fb26806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb26806aed0 0x7fb26806d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2740090b0 tx=0x7fb27401f040 comp rx=0 tx=0).stop 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 msgr2=0x7fb2840cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 0x7fb2840cdb60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2840b2200 tx=0x7fb27c00a620 comp rx=0 tx=0).stop 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 shutdown_connections 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb26806aed0 0x7fb26806d390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2840cd6d0 0x7fb2840cdb60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb270003a30 0x7fb2840b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.111+0000 7fb28b746700 1 --2- 172.21.15.32:0/2343226854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2840c6d40 0x7fb2840c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.112+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 >> 172.21.15.32:0/2343226854 conn(0x7fb2840731a0 msgr2=0x7fb28406ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.112+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 shutdown_connections 2023-11-24T07:59:28.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.112+0000 7fb28b746700 1 -- 172.21.15.32:0/2343226854 wait complete. 2023-11-24T07:59:28.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T07:59:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.267+0000 7f9fe1c6e700 1 Processor -- start 2023-11-24T07:59:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.267+0000 7f9fe1c6e700 1 -- start start 2023-11-24T07:59:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fe1c6e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 0x7f9fdc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fe1c6e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0af950 con 0x7f9fdc06c540 2023-11-24T07:59:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fe1c6e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0afc30 con 0x7f9fdc1446a0 2023-11-24T07:59:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fe1c6e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc144a80 con 0x7f9fdc0adf00 2023-11-24T07:59:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fdbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fdc06c540 0x7f9fdc0b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59836/0 (socket says 172.21.15.32:59836) 2023-11-24T07:59:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fdbfff700 1 -- 172.21.15.32:0/3383829303 learned_addr learned my addr 172.21.15.32:0/3383829303 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fdaffd700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 0x7f9fdc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.268+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764954248 0 0) 0x7f9fdc0af950 con 0x7f9fdc06c540 2023-11-24T07:59:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc80035b0 con 0x7f9fdc06c540 2023-11-24T07:59:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44185164 0 0) 0x7f9fc80035b0 con 0x7f9fdc06c540 2023-11-24T07:59:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 msgr2=0x7f9fdc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 0x7f9fdc06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 >> v1:172.21.15.123:6789/0 conn(0x7f9fdc1446a0 legacy=0x7f9fdc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0aff10 con 0x7f9fdc06c540 2023-11-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f9fd40019f0 con 0x7f9fdc06c540 2023-11-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 >> v1:172.21.15.32:6789/0 conn(0x7f9fdc06c540 legacy=0x7f9fdc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc8003a30 0x7f9fc8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc80043e0 0x7f9fc80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 0x7f9fc8005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc144a80 con 0x7f9fc8003a30 2023-11-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800ba10 con 0x7f9fc80043e0 2023-11-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800bbd0 con 0x7f9fdc0adf00 2023-11-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc8003a30 0x7f9fc8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 0x7f9fc8005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdaffd700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc80043e0 0x7f9fc80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdb7fe700 1 -- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 msgr2=0x7f9fc8005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:28.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc0adf00 0x7f9fc8005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdb7fe700 1 -- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc80043e0 msgr2=0x7f9fc80048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:28.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc80043e0 0x7f9fc80048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.269+0000 7f9fdb7fe700 1 -- 172.21.15.32:0/3383829303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80035b0 con 0x7f9fc8003a30 2023-11-24T07:59:28.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc8003a30 0x7f9fc8003e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0018d0 tx=0x7f9fc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679671236d639143 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00c690 con 0x7f9fc8003a30 2023-11-24T07:59:28.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc00c810 con 0x7f9fc8003a30 2023-11-24T07:59:28.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fda7fc700 1 -- 172.21.15.32:0/3383829303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00cbc0 con 0x7f9fc8003a30 2023-11-24T07:59:28.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/3383829303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc8003a30 msgr2=0x7f9fc8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/3383829303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fc8003a30 0x7f9fc8003e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0018d0 tx=0x7f9fc8005880 comp rx=0 tx=0).stop 2023-11-24T07:59:28.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fdbfff700 1 -- 172.21.15.32:0/3383829303 reap_dead start 2023-11-24T07:59:28.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/3383829303 shutdown_connections 2023-11-24T07:59:28.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/3383829303 >> 172.21.15.32:0/3383829303 conn(0x7f9fdc071b90 msgr2=0x7f9fdc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/3383829303 shutdown_connections 2023-11-24T07:59:28.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.270+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/3383829303 wait complete. 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.271+0000 7f9fe1c6e700 1 Processor -- start 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 -- start start 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc8003a30 0x7f9fdc13a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc09bd70 0x7f9fdc09c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 0x7f9fdc0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc1416e0 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0d01a0 con 0x7f9fc8003a30 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fe1c6e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc140a60 con 0x7f9fdc09bd70 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 0x7f9fdc0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 0x7f9fdc0cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58136/0 (socket says 172.21.15.32:58136) 2023-11-24T07:59:28.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fdbfff700 1 -- 172.21.15.32:0/1554538573 learned_addr learned my addr 172.21.15.32:0/1554538573 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.272+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc8003a30 0x7f9fdc13a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdaffd700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc09bd70 0x7f9fdc09c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdbfff700 1 -- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc09bd70 msgr2=0x7f9fdc09c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc09bd70 0x7f9fdc09c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdbfff700 1 -- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc8003a30 msgr2=0x7f9fdc13a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc8003a30 0x7f9fdc13a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdbfff700 1 -- 172.21.15.32:0/1554538573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc140d40 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 0x7f9fdc0cfc60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f80 tx=0x7f9fd400a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4015410 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd4015570 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc141020 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4015920 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.273+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc13ae20 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.274+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc0753f0 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.275+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fd4015a80 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.275+0000 7f9fd3fff700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9fbc06aed0 0x7f9fbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.275+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9fd408bf20 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.275+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9fbc06aed0 0x7f9fbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.276+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9fbc06aed0 0x7f9fbc06d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc000c00 tx=0x7f9fcc005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.280+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fd4057330 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.563+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9fdc141820 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.565+0000 7f9fd3fff700 1 -- 172.21.15.32:0/1554538573 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9fdc141820 con 0x7f9fdc0cf7d0 2023-11-24T07:59:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9fbc06aed0 msgr2=0x7f9fbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9fbc06aed0 0x7f9fbc06d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc000c00 tx=0x7f9fcc005040 comp rx=0 tx=0).stop 2023-11-24T07:59:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 msgr2=0x7f9fdc0cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 0x7f9fdc0cfc60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f80 tx=0x7f9fd400a850 comp rx=0 tx=0).stop 2023-11-24T07:59:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 shutdown_connections 2023-11-24T07:59:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9fbc06aed0 0x7f9fbc06d390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0cf7d0 0x7f9fdc0cfc60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fc8003a30 0x7f9fdc13a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 --2- 172.21.15.32:0/1554538573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9fdc09bd70 0x7f9fdc09c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 >> 172.21.15.32:0/1554538573 conn(0x7f9fdc071b90 msgr2=0x7f9fdc15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 shutdown_connections 2023-11-24T07:59:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.567+0000 7f9fe1c6e700 1 -- 172.21.15.32:0/1554538573 wait complete. 2023-11-24T07:59:28.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T07:59:28.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.707+0000 7f7cb4794700 1 Processor -- start 2023-11-24T07:59:28.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.707+0000 7f7cb4794700 1 -- start start 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb4794700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cac15dea0 0x7f7cac0d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb4794700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cac15e280 0x7f7cac0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb4794700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac0d0a80 con 0x7f7cac0cf250 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb4794700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac0d2670 con 0x7f7cac15dea0 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb4794700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac06c4a0 con 0x7f7cac15e280 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cac15dea0 0x7f7cac0d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cac15dea0 0x7f7cac0d1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58144/0 (socket says 172.21.15.32:58144) 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 -- 172.21.15.32:0/3411976005 learned_addr learned my addr 172.21.15.32:0/3411976005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb1d2f700 1 --2- 172.21.15.32:0/3411976005 >> v2:172.21.15.123:3300/0 conn(0x7f7cac15e280 0x7f7cac0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 -- 172.21.15.32:0/3411976005 >> v2:172.21.15.123:3300/0 conn(0x7f7cac15e280 msgr2=0x7f7cac0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 --2- 172.21.15.32:0/3411976005 >> v2:172.21.15.123:3300/0 conn(0x7f7cac15e280 0x7f7cac0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 -- 172.21.15.32:0/3411976005 >> v1:172.21.15.32:6789/0 conn(0x7f7cac0cf250 legacy=0x7f7cac06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 -- 172.21.15.32:0/3411976005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0cfeb0 con 0x7f7cac15dea0 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb152e700 1 -- 172.21.15.32:0/3411976005 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269322437 0 0) 0x7f7cac0d0a80 con 0x7f7cac0cf250 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.708+0000 7f7cb2530700 1 --2- 172.21.15.32:0/3411976005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac15dea0 0x7f7cac0d1e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7cac09b7f0 tx=0x7f7ca8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd51b934cb82c0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb152e700 1 -- 172.21.15.32:0/3411976005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8011830 con 0x7f7cac15dea0 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb152e700 1 -- 172.21.15.32:0/3411976005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca8011990 con 0x7f7cac15dea0 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb152e700 1 -- 172.21.15.32:0/3411976005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8011d20 con 0x7f7cac15dea0 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 -- 172.21.15.32:0/3411976005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac15dea0 msgr2=0x7f7cac0d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 --2- 172.21.15.32:0/3411976005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac15dea0 0x7f7cac0d1e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7cac09b7f0 tx=0x7f7ca8009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 -- 172.21.15.32:0/3411976005 shutdown_connections 2023-11-24T07:59:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 --2- 172.21.15.32:0/3411976005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac15dea0 0x7f7cac0d1e40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 --2- 172.21.15.32:0/3411976005 >> v2:172.21.15.123:3300/0 conn(0x7f7cac15e280 0x7f7cac0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 -- 172.21.15.32:0/3411976005 >> 172.21.15.32:0/3411976005 conn(0x7f7cac073130 msgr2=0x7f7cac09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 -- 172.21.15.32:0/3411976005 shutdown_connections 2023-11-24T07:59:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.709+0000 7f7cb4794700 1 -- 172.21.15.32:0/3411976005 wait complete. 2023-11-24T07:59:28.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.710+0000 7f7cb4794700 1 Processor -- start 2023-11-24T07:59:28.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.710+0000 7f7cb4794700 1 -- start start 2023-11-24T07:59:28.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb4794700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 0x7f7cac0c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb4794700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cac15dea0 0x7f7cac0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb4794700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cac1f76e0 0x7f7cac15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb4794700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c6160 con 0x7f7cac0cf250 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb4794700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c6840 con 0x7f7cac1f76e0 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb4794700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac09f0a0 con 0x7f7cac15dea0 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb2530700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 0x7f7cac0c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb2530700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 0x7f7cac0c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58150/0 (socket says 172.21.15.32:58150) 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb2530700 1 -- 172.21.15.32:0/1100549798 learned_addr learned my addr 172.21.15.32:0/1100549798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.711+0000 7f7cb2d31700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cac1f76e0 0x7f7cac15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2530700 1 -- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cac15dea0 msgr2=0x7f7cac0c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2530700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cac15dea0 0x7f7cac0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2530700 1 -- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cac1f76e0 msgr2=0x7f7cac15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2530700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cac1f76e0 0x7f7cac15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2530700 1 -- 172.21.15.32:0/1100549798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8009550 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2d31700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cac1f76e0 0x7f7cac15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb2530700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 0x7f7cac0c5c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009870 tx=0x7f7ca8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80044b0 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca801f070 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac0c9d90 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80038c0 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.712+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac0ca350 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.713+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac0c5570 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.713+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ca8021030 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.714+0000 7f7ca2ffd700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9006aed0 0x7f7c9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.714+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ca808a620 con 0x7f7cac0cf250 2023-11-24T07:59:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.714+0000 7f7cb1d2f700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9006aed0 0x7f7c9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.714+0000 7f7cb1d2f700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9006aed0 0x7f7c9006d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cac09f3a0 tx=0x7f7c9c0095a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:28.718+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ca8055bd0 con 0x7f7cac0cf250 2023-11-24T07:59:29.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.001+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f7cac067f60 con 0x7f7cac0cf250 2023-11-24T07:59:29.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.003+0000 7f7ca2ffd700 1 -- 172.21.15.32:0/1100549798 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7ca80150a0 con 0x7f7cac0cf250 2023-11-24T07:59:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9006aed0 msgr2=0x7f7c9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9006aed0 0x7f7c9006d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cac09f3a0 tx=0x7f7c9c0095a0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 msgr2=0x7f7cac0c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 0x7f7cac0c5c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009870 tx=0x7f7ca8004750 comp rx=0 tx=0).stop 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 shutdown_connections 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7c9006aed0 0x7f7c9006d390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cac1f76e0 0x7f7cac15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac0cf250 0x7f7cac0c5c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 --2- 172.21.15.32:0/1100549798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7cac15dea0 0x7f7cac0c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 >> 172.21.15.32:0/1100549798 conn(0x7f7cac073130 msgr2=0x7f7cac06fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 shutdown_connections 2023-11-24T07:59:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.005+0000 7f7cb4794700 1 -- 172.21.15.32:0/1100549798 wait complete. 2023-11-24T07:59:29.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T07:59:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.144+0000 7fa92a998700 1 Processor -- start 2023-11-24T07:59:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.144+0000 7fa92a998700 1 -- start start 2023-11-24T07:59:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.145+0000 7fa92a998700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9240adf00 0x7fa92406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.145+0000 7fa92a998700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 0x7fa9240b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.145+0000 7fa92a998700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9240af950 con 0x7fa9241446a0 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.145+0000 7fa92a998700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa924144a80 con 0x7fa92406c540 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.145+0000 7fa92a998700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92406c9b0 con 0x7fa9240adf00 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa928f35700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 0x7fa9240b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa9237fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9240adf00 0x7fa92406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa928f35700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 0x7fa9240b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:42312/0 (socket says 172.21.15.32:42312) 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa928f35700 1 -- 172.21.15.32:0/1366264637 learned_addr learned my addr 172.21.15.32:0/1366264637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa9237fe700 1 -- 172.21.15.32:0/1366264637 >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 msgr2=0x7fa9240b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa9237fe700 1 --2- 172.21.15.32:0/1366264637 >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 0x7fa9240b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa9237fe700 1 -- 172.21.15.32:0/1366264637 >> v1:172.21.15.123:6789/0 conn(0x7fa9241446a0 legacy=0x7fa9240b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa9237fe700 1 -- 172.21.15.32:0/1366264637 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240afc30 con 0x7fa9240adf00 2023-11-24T07:59:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa928f35700 1 --2- 172.21.15.32:0/1366264637 >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 0x7fa9240b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa9237fe700 1 --2- 172.21.15.32:0/1366264637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9240adf00 0x7fa92406c000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa914009840 tx=0x7fa914009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dedbd4b06965419b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.146+0000 7fa922ffd700 1 -- 172.21.15.32:0/1366264637 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914011830 con 0x7fa9240adf00 2023-11-24T07:59:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa922ffd700 1 -- 172.21.15.32:0/1366264637 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914011990 con 0x7fa9240adf00 2023-11-24T07:59:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa922ffd700 1 -- 172.21.15.32:0/1366264637 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914011d20 con 0x7fa9240adf00 2023-11-24T07:59:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 -- 172.21.15.32:0/1366264637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9240adf00 msgr2=0x7fa92406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 --2- 172.21.15.32:0/1366264637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9240adf00 0x7fa92406c000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa914009840 tx=0x7fa914009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 -- 172.21.15.32:0/1366264637 shutdown_connections 2023-11-24T07:59:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 --2- 172.21.15.32:0/1366264637 >> v2:172.21.15.123:3300/0 conn(0x7fa92406c540 0x7fa9240b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 --2- 172.21.15.32:0/1366264637 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9240adf00 0x7fa92406c000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 -- 172.21.15.32:0/1366264637 >> 172.21.15.32:0/1366264637 conn(0x7fa924071b90 msgr2=0x7fa924071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 -- 172.21.15.32:0/1366264637 shutdown_connections 2023-11-24T07:59:29.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.147+0000 7fa92a998700 1 -- 172.21.15.32:0/1366264637 wait complete. 2023-11-24T07:59:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.149+0000 7fa92a998700 1 Processor -- start 2023-11-24T07:59:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.149+0000 7fa92a998700 1 -- start start 2023-11-24T07:59:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa92a998700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92406c540 0x7fa924156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa92a998700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 0x7fa92413d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa92a998700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9241446a0 0x7fa92413eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa92a998700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240cdd60 con 0x7fa9240adf00 2023-11-24T07:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa92a998700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92413e0b0 con 0x7fa9241446a0 2023-11-24T07:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa92a998700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92413ddd0 con 0x7fa92406c540 2023-11-24T07:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 0x7fa92413d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 0x7fa92413d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58166/0 (socket says 172.21.15.32:58166) 2023-11-24T07:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 -- 172.21.15.32:0/1954178518 learned_addr learned my addr 172.21.15.32:0/1954178518 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa923fff700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92406c540 0x7fa924156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa928f35700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9241446a0 0x7fa92413eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 -- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92406c540 msgr2=0x7fa924156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92406c540 0x7fa924156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 -- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9241446a0 msgr2=0x7fa92413eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9241446a0 0x7fa92413eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 -- 172.21.15.32:0/1954178518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c009b30 con 0x7fa9240adf00 2023-11-24T07:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.150+0000 7fa9237fe700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 0x7fa92413d460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa9140090b0 tx=0x7fa914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.151+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91401a760 con 0x7fa9240adf00 2023-11-24T07:59:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.151+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91401a8c0 con 0x7fa9240adf00 2023-11-24T07:59:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.151+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91401ac70 con 0x7fa9240adf00 2023-11-24T07:59:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.151+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9140094a0 con 0x7fa9240adf00 2023-11-24T07:59:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.151+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9240cd9f0 con 0x7fa9240adf00 2023-11-24T07:59:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.152+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9240753f0 con 0x7fa9240adf00 2023-11-24T07:59:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.152+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa914022070 con 0x7fa9240adf00 2023-11-24T07:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.152+0000 7fa920ff9700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa90406aed0 0x7fa90406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.153+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa91408a090 con 0x7fa9240adf00 2023-11-24T07:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.153+0000 7fa923fff700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa90406aed0 0x7fa90406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.153+0000 7fa923fff700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa90406aed0 0x7fa90406d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa92413fec0 tx=0x7fa90c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.157+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa914055410 con 0x7fa9240adf00 2023-11-24T07:59:29.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.444+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa9240cdb80 con 0x7fa9240adf00 2023-11-24T07:59:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.446+0000 7fa920ff9700 1 -- 172.21.15.32:0/1954178518 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa914058a30 con 0x7fa9240adf00 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa90406aed0 msgr2=0x7fa90406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa90406aed0 0x7fa90406d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa92413fec0 tx=0x7fa90c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 msgr2=0x7fa92413d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 0x7fa92413d460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa9140090b0 tx=0x7fa914004750 comp rx=0 tx=0).stop 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 shutdown_connections 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa90406aed0 0x7fa90406d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9241446a0 0x7fa92413eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa92406c540 0x7fa924156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 --2- 172.21.15.32:0/1954178518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9240adf00 0x7fa92413d460 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 >> 172.21.15.32:0/1954178518 conn(0x7fa924071b90 msgr2=0x7fa9240b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 shutdown_connections 2023-11-24T07:59:29.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.448+0000 7fa92a998700 1 -- 172.21.15.32:0/1954178518 wait complete. 2023-11-24T07:59:29.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T07:59:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.586+0000 7f93f1128700 1 Processor -- start 2023-11-24T07:59:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.586+0000 7f93f1128700 1 -- start start 2023-11-24T07:59:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.586+0000 7f93f1128700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.586+0000 7f93f1128700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ec0af950 con 0x7f93ec06c710 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.586+0000 7f93f1128700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ec0afc30 con 0x7f93ec0adf00 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.586+0000 7f93f1128700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec144a80 con 0x7f93ec1446a0 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93eb59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93ec06c710 0x7f93ec0b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59888/0 (socket says 172.21.15.32:59888) 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93eb59e700 1 -- 172.21.15.32:0/2386315461 learned_addr learned my addr 172.21.15.32:0/2386315461 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93ead9d700 1 --2- 172.21.15.32:0/2386315461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078042887 0 0) 0x7f93ec0af950 con 0x7f93ec06c710 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93d40035b0 con 0x7f93ec06c710 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825027286 0 0) 0x7f93ec0afc30 con 0x7f93ec0adf00 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ec0af950 con 0x7f93ec0adf00 2023-11-24T07:59:29.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93ead9d700 1 -- 172.21.15.32:0/2386315461 >> v1:172.21.15.123:6789/0 conn(0x7f93ec0adf00 legacy=0x7f93ec06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93ead9d700 1 -- 172.21.15.32:0/2386315461 >> v1:172.21.15.32:6789/0 conn(0x7f93ec06c710 legacy=0x7f93ec0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180938827 0 0) 0x7f93d40035b0 con 0x7f93ec06c710 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93ead9d700 1 -- 172.21.15.32:0/2386315461 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec0aff10 con 0x7f93ec1446a0 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.587+0000 7f93ead9d700 1 --2- 172.21.15.32:0/2386315461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec0b5ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009870 tx=0x7f93dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957fdc4524af5fd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e40019f0 con 0x7f93ec1446a0 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc004030 con 0x7f93ec1446a0 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93e9d9b700 1 -- 172.21.15.32:0/2386315461 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0043c0 con 0x7f93ec1446a0 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 -- 172.21.15.32:0/2386315461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 msgr2=0x7f93ec0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 --2- 172.21.15.32:0/2386315461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec0b5ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009870 tx=0x7f93dc009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 -- 172.21.15.32:0/2386315461 shutdown_connections 2023-11-24T07:59:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 --2- 172.21.15.32:0/2386315461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec0b5ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 -- 172.21.15.32:0/2386315461 >> 172.21.15.32:0/2386315461 conn(0x7f93ec071b90 msgr2=0x7f93ec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 -- 172.21.15.32:0/2386315461 shutdown_connections 2023-11-24T07:59:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.588+0000 7f93f1128700 1 -- 172.21.15.32:0/2386315461 wait complete. 2023-11-24T07:59:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 Processor -- start 2023-11-24T07:59:29.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 -- start start 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec06c710 0x7f93ec0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 0x7f93ec1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0cec80 con 0x7f93ec0adf00 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0ce650 con 0x7f93ec06c710 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93f1128700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1f43d0 con 0x7f93ec1446a0 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.590+0000 7f93ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 0x7f93ec1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 0x7f93ec1f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58180/0 (socket says 172.21.15.32:58180) 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 -- 172.21.15.32:0/2698707502 learned_addr learned my addr 172.21.15.32:0/2698707502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ead9d700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec06c710 0x7f93ec0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93eb59e700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 -- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 msgr2=0x7f93ec1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 -- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec06c710 msgr2=0x7f93ec0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec06c710 0x7f93ec0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 -- 172.21.15.32:0/2698707502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc0094a0 con 0x7f93ec0adf00 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93ea59c700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 0x7f93ec1f3850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c4f0 tx=0x7f93e000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00052d0 con 0x7f93ec0adf00 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0005430 con 0x7f93ec0adf00 2023-11-24T07:59:29.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec1f46b0 con 0x7f93ec0adf00 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0003880 con 0x7f93ec0adf00 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.591+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec0d02a0 con 0x7f93ec0adf00 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.592+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec0753f0 con 0x7f93ec0adf00 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.593+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93e00039e0 con 0x7f93ec0adf00 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.593+0000 7f93d37fe700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f93d806aed0 0x7f93d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.593+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93e00885c0 con 0x7f93ec0adf00 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.593+0000 7f93ead9d700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f93d806aed0 0x7f93d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.594+0000 7f93ead9d700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f93d806aed0 0x7f93d806d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097c0 tx=0x7f93dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.598+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93e00539f0 con 0x7f93ec0adf00 2023-11-24T07:59:29.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.883+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f93ec067f60 con 0x7f93ec0adf00 2023-11-24T07:59:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.884+0000 7f93d37fe700 1 -- 172.21.15.32:0/2698707502 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f93e0057010 con 0x7f93ec0adf00 2023-11-24T07:59:29.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f93d806aed0 msgr2=0x7f93d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f93d806aed0 0x7f93d806d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097c0 tx=0x7f93dc008030 comp rx=0 tx=0).stop 2023-11-24T07:59:29.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 msgr2=0x7f93ec1f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 0x7f93ec1f3850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c4f0 tx=0x7f93e000c800 comp rx=0 tx=0).stop 2023-11-24T07:59:29.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 shutdown_connections 2023-11-24T07:59:29.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f93d806aed0 0x7f93d806d390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93ec1446a0 0x7f93ec1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93ec06c710 0x7f93ec0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.886+0000 7f93f1128700 1 --2- 172.21.15.32:0/2698707502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec0adf00 0x7f93ec1f3850 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.887+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 >> 172.21.15.32:0/2698707502 conn(0x7f93ec071b90 msgr2=0x7f93ec06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.887+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 shutdown_connections 2023-11-24T07:59:29.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:29.887+0000 7f93f1128700 1 -- 172.21.15.32:0/2698707502 wait complete. 2023-11-24T07:59:29.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T07:59:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.024+0000 7f0051ac1700 1 Processor -- start 2023-11-24T07:59:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.024+0000 7f0051ac1700 1 -- start start 2023-11-24T07:59:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f0051ac1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f004c15be00 0x7f004c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f0051ac1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f004c15c1e0 0x7f004c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f0051ac1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f0051ac1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c0b00e0 con 0x7f004c15be00 2023-11-24T07:59:30.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f0051ac1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c0af4d0 con 0x7f004c15c1e0 2023-11-24T07:59:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f0051ac1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0af7b0 con 0x7f004c0af030 2023-11-24T07:59:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f004b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f004c15be00 0x7f004c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f004b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f004c15be00 0x7f004c0cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58186/0 (socket says 172.21.15.32:58186) 2023-11-24T07:59:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.025+0000 7f004b7fe700 1 -- 172.21.15.32:0/28354008 learned_addr learned my addr 172.21.15.32:0/28354008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:30.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004bfff700 1 --2- 172.21.15.32:0/28354008 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004affd700 1 --2- 172.21.15.32:0/28354008 >> v2:172.21.15.123:3300/0 conn(0x7f004c15c1e0 0x7f004c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004b7fe700 1 -- 172.21.15.32:0/28354008 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 msgr2=0x7f004c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004b7fe700 1 --2- 172.21.15.32:0/28354008 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c0d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004b7fe700 1 -- 172.21.15.32:0/28354008 >> v2:172.21.15.123:3300/0 conn(0x7f004c15c1e0 msgr2=0x7f004c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004b7fe700 1 --2- 172.21.15.32:0/28354008 >> v2:172.21.15.123:3300/0 conn(0x7f004c15c1e0 0x7f004c0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004b7fe700 1 -- 172.21.15.32:0/28354008 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0afa90 con 0x7f004c15be00 2023-11-24T07:59:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004b7fe700 1 --2- 172.21.15.32:0/28354008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0cc660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f004c0c5540 tx=0x7f003c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb3c54d69f175d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004a7fc700 1 -- 172.21.15.32:0/28354008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c011830 con 0x7f004c15be00 2023-11-24T07:59:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004a7fc700 1 -- 172.21.15.32:0/28354008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c011990 con 0x7f004c15be00 2023-11-24T07:59:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.026+0000 7f004a7fc700 1 -- 172.21.15.32:0/28354008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c011d20 con 0x7f004c15be00 2023-11-24T07:59:30.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 -- 172.21.15.32:0/28354008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 msgr2=0x7f004c0cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 --2- 172.21.15.32:0/28354008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0cc660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f004c0c5540 tx=0x7f003c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 -- 172.21.15.32:0/28354008 shutdown_connections 2023-11-24T07:59:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 --2- 172.21.15.32:0/28354008 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 --2- 172.21.15.32:0/28354008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0cc660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 --2- 172.21.15.32:0/28354008 >> v2:172.21.15.123:3300/0 conn(0x7f004c15c1e0 0x7f004c0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 -- 172.21.15.32:0/28354008 >> 172.21.15.32:0/28354008 conn(0x7f004c09aa00 msgr2=0x7f004c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 -- 172.21.15.32:0/28354008 shutdown_connections 2023-11-24T07:59:30.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.027+0000 7f0051ac1700 1 -- 172.21.15.32:0/28354008 wait complete. 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.028+0000 7f0051ac1700 1 Processor -- start 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.029+0000 7f0051ac1700 1 -- start start 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.029+0000 7f0051ac1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.029+0000 7f0051ac1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.029+0000 7f0051ac1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004c15c1e0 0x7f004c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0051ac1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0cab90 con 0x7f004c15be00 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0051ac1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c151a00 con 0x7f004c15c1e0 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0051ac1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0c6b80 con 0x7f004c0af030 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58194/0 (socket says 172.21.15.32:58194) 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 -- 172.21.15.32:0/1802060706 learned_addr learned my addr 172.21.15.32:0/1802060706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004bfff700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004c15c1e0 0x7f004c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004b7fe700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 -- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 msgr2=0x7f004c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 -- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004c15c1e0 msgr2=0x7f004c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004c15c1e0 0x7f004c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 -- 172.21.15.32:0/1802060706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009550 con 0x7f004c15be00 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f004affd700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0ca650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f003400e870 tx=0x7f003400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034009590 con 0x7f004c15be00 2023-11-24T07:59:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c15dc90 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c15e250 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034013680 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.030+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00340139f0 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.031+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c0c6d40 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.032+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0034013b50 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.032+0000 7f0043fff700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f002c06aed0 0x7f002c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.032+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0034089710 con 0x7f004c15be00 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.032+0000 7f004b7fe700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f002c06aed0 0x7f002c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.033+0000 7f004b7fe700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f002c06aed0 0x7f002c06d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f003c009c10 tx=0x7f003c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.037+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0034054a90 con 0x7f004c15be00 2023-11-24T07:59:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.319+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f004c067f60 con 0x7f004c15be00 2023-11-24T07:59:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.321+0000 7f0043fff700 1 -- 172.21.15.32:0/1802060706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f00340580b0 con 0x7f004c15be00 2023-11-24T07:59:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f002c06aed0 msgr2=0x7f002c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f002c06aed0 0x7f002c06d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f003c009c10 tx=0x7f003c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 msgr2=0x7f004c0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0ca650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f003400e870 tx=0x7f003400eb80 comp rx=0 tx=0).stop 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 shutdown_connections 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f002c06aed0 0x7f002c06d390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f004c15c1e0 0x7f004c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f004c0af030 0x7f004c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 --2- 172.21.15.32:0/1802060706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f004c15be00 0x7f004c0ca650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 >> 172.21.15.32:0/1802060706 conn(0x7f004c09aa00 msgr2=0x7f004c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 shutdown_connections 2023-11-24T07:59:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.323+0000 7f0051ac1700 1 -- 172.21.15.32:0/1802060706 wait complete. 2023-11-24T07:59:30.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T07:59:30.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.461+0000 7fbf6c5aa700 1 Processor -- start 2023-11-24T07:59:30.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.461+0000 7fbf6c5aa700 1 -- start start 2023-11-24T07:59:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.462+0000 7fbf6c5aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf640b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.462+0000 7fbf6c5aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640ae430 con 0x7fbf6415e080 2023-11-24T07:59:30.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.462+0000 7fbf6c5aa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640ae710 con 0x7fbf640af4d0 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.462+0000 7fbf6c5aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640af0b0 con 0x7fbf6406c710 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf6a346700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf6415e080 0x7fbf6406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59912/0 (socket says 172.21.15.32:59912) 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf6a346700 1 -- 172.21.15.32:0/2302165630 learned_addr learned my addr 172.21.15.32:0/2302165630 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69b45700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf640b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536087879 0 0) 0x7fbf640ae430 con 0x7fbf6415e080 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf500035b0 con 0x7fbf6415e080 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621100367 0 0) 0x7fbf640ae710 con 0x7fbf640af4d0 2023-11-24T07:59:30.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf640ae430 con 0x7fbf640af4d0 2023-11-24T07:59:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051590941 0 0) 0x7fbf500035b0 con 0x7fbf6415e080 2023-11-24T07:59:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 msgr2=0x7fbf640b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf640b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 >> v1:172.21.15.123:6789/0 conn(0x7fbf640af4d0 legacy=0x7fbf6406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640ae9f0 con 0x7fbf6415e080 2023-11-24T07:59:30.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69b45700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf640b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:30.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fbf60001670 con 0x7fbf6415e080 2023-11-24T07:59:30.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 >> v1:172.21.15.32:6789/0 conn(0x7fbf6415e080 legacy=0x7fbf6406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf50003bc0 0x7fbf50003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50004d90 0x7fbf50005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640af0b0 con 0x7fbf50003bc0 2023-11-24T07:59:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000bac0 con 0x7fbf50004d90 2023-11-24T07:59:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000bc00 con 0x7fbf6406c710 2023-11-24T07:59:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf6a346700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf50003bc0 0x7fbf50003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf6ab47700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50004d90 0x7fbf50005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.463+0000 7fbf69b45700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6a346700 1 -- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 msgr2=0x7fbf50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6a346700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6a346700 1 -- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50004d90 msgr2=0x7fbf50005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6a346700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50004d90 0x7fbf50005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6a346700 1 -- 172.21.15.32:0/2302165630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580094a0 con 0x7fbf50003bc0 2023-11-24T07:59:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf69b45700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf6406c710 0x7fbf50004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6a346700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf50003bc0 0x7fbf50003fc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c240 tx=0x7fbf50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ca026a93a2f774 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000cda0 con 0x7fbf50003bc0 2023-11-24T07:59:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60001af0 con 0x7fbf50003bc0 2023-11-24T07:59:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf69344700 1 -- 172.21.15.32:0/2302165630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60005ba0 con 0x7fbf50003bc0 2023-11-24T07:59:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/2302165630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf50003bc0 msgr2=0x7fbf50003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/2302165630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf50003bc0 0x7fbf50003fc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c240 tx=0x7fbf50005880 comp rx=0 tx=0).stop 2023-11-24T07:59:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.464+0000 7fbf6ab47700 1 -- 172.21.15.32:0/2302165630 reap_dead start 2023-11-24T07:59:30.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.465+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/2302165630 shutdown_connections 2023-11-24T07:59:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.465+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/2302165630 >> 172.21.15.32:0/2302165630 conn(0x7fbf64071b90 msgr2=0x7fbf64071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.465+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/2302165630 shutdown_connections 2023-11-24T07:59:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.465+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/2302165630 wait complete. 2023-11-24T07:59:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.466+0000 7fbf6c5aa700 1 Processor -- start 2023-11-24T07:59:30.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.466+0000 7fbf6c5aa700 1 -- start start 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50003bc0 0x7fbf64157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 0x7fbf6415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf64153bb0 0x7fbf64154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf641545a0 con 0x7fbf64158870 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64159360 con 0x7fbf50003bc0 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64159080 con 0x7fbf64153bb0 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 0x7fbf6415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 0x7fbf6415fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58214/0 (socket says 172.21.15.32:58214) 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 -- 172.21.15.32:0/66519299 learned_addr learned my addr 172.21.15.32:0/66519299 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:30.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6a346700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50003bc0 0x7fbf64157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6ab47700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf64153bb0 0x7fbf64154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 -- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf64153bb0 msgr2=0x7fbf64154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf64153bb0 0x7fbf64154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 -- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50003bc0 msgr2=0x7fbf64157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50003bc0 0x7fbf64157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 -- 172.21.15.32:0/66519299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640cd6c0 con 0x7fbf64158870 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf69b45700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 0x7fbf6415fc00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbf58010c30 tx=0x7fbf58010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800e8d0 con 0x7fbf64158870 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5800ea30 con 0x7fbf64158870 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf640cd9a0 con 0x7fbf64158870 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800eda0 con 0x7fbf64158870 2023-11-24T07:59:30.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.467+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf64154750 con 0x7fbf64158870 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.468+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf640753f0 con 0x7fbf64158870 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.469+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf58016460 con 0x7fbf64158870 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.469+0000 7fbf56ffd700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf4806aed0 0x7fbf4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.469+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf580896b0 con 0x7fbf64158870 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.469+0000 7fbf6a346700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf4806aed0 0x7fbf4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.470+0000 7fbf6a346700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf4806aed0 0x7fbf4806d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf60001eb0 tx=0x7fbf6000a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.474+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf58054ab0 con 0x7fbf64158870 2023-11-24T07:59:30.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.761+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fbf64067f60 con 0x7fbf64158870 2023-11-24T07:59:30.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.763+0000 7fbf56ffd700 1 -- 172.21.15.32:0/66519299 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbf580580d0 con 0x7fbf64158870 2023-11-24T07:59:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf4806aed0 msgr2=0x7fbf4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf4806aed0 0x7fbf4806d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf60001eb0 tx=0x7fbf6000a000 comp rx=0 tx=0).stop 2023-11-24T07:59:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 msgr2=0x7fbf6415fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 0x7fbf6415fc00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbf58010c30 tx=0x7fbf58010f40 comp rx=0 tx=0).stop 2023-11-24T07:59:30.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 shutdown_connections 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbf4806aed0 0x7fbf4806d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbf64153bb0 0x7fbf64154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf50003bc0 0x7fbf64157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 --2- 172.21.15.32:0/66519299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64158870 0x7fbf6415fc00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 >> 172.21.15.32:0/66519299 conn(0x7fbf64071b90 msgr2=0x7fbf640af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 shutdown_connections 2023-11-24T07:59:30.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.766+0000 7fbf6c5aa700 1 -- 172.21.15.32:0/66519299 wait complete. 2023-11-24T07:59:30.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:30.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.903+0000 7f1df8f6e700 1 Processor -- start 2023-11-24T07:59:30.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.903+0000 7f1df8f6e700 1 -- start start 2023-11-24T07:59:30.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df8f6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df406c710 0x7f1df40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df8f6e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df415c4b0 con 0x7f1df40b49a0 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df8f6e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df415c790 con 0x7f1df4152cc0 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df8f6e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df415ca70 con 0x7f1df406c710 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df406c710 0x7f1df40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df406c710 0x7f1df40cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58222/0 (socket says 172.21.15.32:58222) 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1df40b49a0 0x7f1df40d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59936/0 (socket says 172.21.15.32:59936) 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 -- 172.21.15.32:0/3197582260 learned_addr learned my addr 172.21.15.32:0/3197582260 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 -- 172.21.15.32:0/3197582260 >> v1:172.21.15.123:6789/0 conn(0x7f1df4152cc0 legacy=0x7f1df406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 -- 172.21.15.32:0/3197582260 >> v1:172.21.15.32:6789/0 conn(0x7f1df40b49a0 legacy=0x7f1df40d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 -- 172.21.15.32:0/3197582260 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df415cd50 con 0x7f1df406c710 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df1d9b700 1 --2- 172.21.15.32:0/3197582260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df406c710 0x7f1df40cff20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1de40098f0 tx=0x7f1de4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4929e5f3998b5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df159a700 1 -- 172.21.15.32:0/3197582260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400b8a0 con 0x7f1df406c710 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df159a700 1 -- 172.21.15.32:0/3197582260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de400ba00 con 0x7f1df406c710 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.905+0000 7f1df159a700 1 -- 172.21.15.32:0/3197582260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400bd90 con 0x7f1df406c710 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 -- 172.21.15.32:0/3197582260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df406c710 msgr2=0x7f1df40cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/3197582260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df406c710 0x7f1df40cff20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1de40098f0 tx=0x7f1de4009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 -- 172.21.15.32:0/3197582260 shutdown_connections 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/3197582260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df406c710 0x7f1df40cff20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 -- 172.21.15.32:0/3197582260 >> 172.21.15.32:0/3197582260 conn(0x7f1df4071b90 msgr2=0x7f1df4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 -- 172.21.15.32:0/3197582260 shutdown_connections 2023-11-24T07:59:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.906+0000 7f1df8f6e700 1 -- 172.21.15.32:0/3197582260 wait complete. 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.908+0000 7f1df8f6e700 1 Processor -- start 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.908+0000 7f1df8f6e700 1 -- start start 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df8f6e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1df406c710 0x7f1df4140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df8f6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 0x7f1df41414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df8f6e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1df4152cc0 0x7f1df4143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df8f6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df4142430 con 0x7f1df40b49a0 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df8f6e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df409aa00 con 0x7f1df406c710 2023-11-24T07:59:30.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df8f6e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df409ace0 con 0x7f1df4152cc0 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 0x7f1df41414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 0x7f1df41414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58228/0 (socket says 172.21.15.32:58228) 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 -- 172.21.15.32:0/2250070512 learned_addr learned my addr 172.21.15.32:0/2250070512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df259c700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1df406c710 0x7f1df4140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df2d9d700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1df4152cc0 0x7f1df4143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 -- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1df4152cc0 msgr2=0x7f1df4143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1df4152cc0 0x7f1df4143ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 -- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1df406c710 msgr2=0x7f1df4140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1df406c710 0x7f1df4140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 -- 172.21.15.32:0/2250070512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4009550 con 0x7f1df40b49a0 2023-11-24T07:59:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.909+0000 7f1df1d9b700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 0x7f1df41414a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.910+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40044d0 con 0x7f1df40b49a0 2023-11-24T07:59:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.910+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de401f070 con 0x7f1df40b49a0 2023-11-24T07:59:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.910+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df4141d70 con 0x7f1df40b49a0 2023-11-24T07:59:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.910+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40038c0 con 0x7f1df40b49a0 2023-11-24T07:59:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.910+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df409c410 con 0x7f1df40b49a0 2023-11-24T07:59:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.911+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df40753f0 con 0x7f1df40b49a0 2023-11-24T07:59:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.911+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1de4003a20 con 0x7f1df40b49a0 2023-11-24T07:59:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.912+0000 7f1de2ffd700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dd406aed0 0x7f1dd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.912+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1de4021070 con 0x7f1df40b49a0 2023-11-24T07:59:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.912+0000 7f1df259c700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dd406aed0 0x7f1dd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.912+0000 7f1df259c700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dd406aed0 0x7f1dd406d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1de8001800 tx=0x7f1de8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:30.916+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1de4055f10 con 0x7f1df40b49a0 2023-11-24T07:59:31.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.116+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1df4141bd0 con 0x7f1df40b49a0 2023-11-24T07:59:31.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.117+0000 7f1de2ffd700 1 -- 172.21.15.32:0/2250070512 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12845 (secure 0 0 0) 0x7f1df4141bd0 con 0x7f1df40b49a0 2023-11-24T07:59:31.119 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:31.119 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:27.667457+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:31.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dd406aed0 msgr2=0x7f1dd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dd406aed0 0x7f1dd406d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1de8001800 tx=0x7f1de8009040 comp rx=0 tx=0).stop 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 msgr2=0x7f1df41414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 0x7f1df41414a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4004750 comp rx=0 tx=0).stop 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 shutdown_connections 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1dd406aed0 0x7f1dd406d390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1df4152cc0 0x7f1df4143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1df406c710 0x7f1df4140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 --2- 172.21.15.32:0/2250070512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df40b49a0 0x7f1df41414a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 >> 172.21.15.32:0/2250070512 conn(0x7f1df4071b90 msgr2=0x7f1df406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 shutdown_connections 2023-11-24T07:59:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.119+0000 7f1df8f6e700 1 -- 172.21.15.32:0/2250070512 wait complete. 2023-11-24T07:59:31.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:31.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.260+0000 7f331b8fe700 1 Processor -- start 2023-11-24T07:59:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.261+0000 7f331b8fe700 1 -- start start 2023-11-24T07:59:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.261+0000 7f331b8fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f33140cf610 0x7f33140add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331b8fe700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331406b080 con 0x7f331415f9e0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331b8fe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331415c290 con 0x7f3314073f90 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331b8fe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33140cf9f0 con 0x7f33140cf610 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f3318e99700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f331415f9e0 0x7f33140b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59946/0 (socket says 172.21.15.32:59946) 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f3318e99700 1 -- 172.21.15.32:0/734769553 learned_addr learned my addr 172.21.15.32:0/734769553 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331969a700 1 --2- 172.21.15.32:0/734769553 >> v2:172.21.15.123:3300/0 conn(0x7f33140cf610 0x7f33140add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f330bfff700 1 -- 172.21.15.32:0/734769553 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424156160 0 0) 0x7f331406b080 con 0x7f331415f9e0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f330bfff700 1 -- 172.21.15.32:0/734769553 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33000035b0 con 0x7f331415f9e0 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331969a700 1 -- 172.21.15.32:0/734769553 >> v1:172.21.15.123:6789/0 conn(0x7f3314073f90 legacy=0x7f331406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331969a700 1 -- 172.21.15.32:0/734769553 >> v1:172.21.15.32:6789/0 conn(0x7f331415f9e0 legacy=0x7f33140b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331969a700 1 -- 172.21.15.32:0/734769553 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331415ff80 con 0x7f33140cf610 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.262+0000 7f331969a700 1 --2- 172.21.15.32:0/734769553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33140cf610 0x7f33140add40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3304009870 tx=0x7f3304009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92aacd5c4c78143a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f330bfff700 1 -- 172.21.15.32:0/734769553 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304011830 con 0x7f33140cf610 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f330bfff700 1 -- 172.21.15.32:0/734769553 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3304011990 con 0x7f33140cf610 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f330bfff700 1 -- 172.21.15.32:0/734769553 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304011d20 con 0x7f33140cf610 2023-11-24T07:59:31.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 -- 172.21.15.32:0/734769553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33140cf610 msgr2=0x7f33140add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 --2- 172.21.15.32:0/734769553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33140cf610 0x7f33140add40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3304009870 tx=0x7f3304009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 -- 172.21.15.32:0/734769553 shutdown_connections 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 --2- 172.21.15.32:0/734769553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33140cf610 0x7f33140add40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 -- 172.21.15.32:0/734769553 >> 172.21.15.32:0/734769553 conn(0x7f33140d0bf0 msgr2=0x7f33140d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 -- 172.21.15.32:0/734769553 shutdown_connections 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.263+0000 7f331b8fe700 1 -- 172.21.15.32:0/734769553 wait complete. 2023-11-24T07:59:31.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.264+0000 7f331b8fe700 1 Processor -- start 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.264+0000 7f331b8fe700 1 -- start start 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331b8fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3314073f90 0x7f33141595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331b8fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 0x7f33140c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331b8fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331415f9e0 0x7f331409cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331b8fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331409aa00 con 0x7f33140cf610 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331b8fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331409aca0 con 0x7f331415f9e0 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331b8fe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331409af80 con 0x7f3314073f90 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f3318e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 0x7f33140c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f3318e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 0x7f33140c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58242/0 (socket says 172.21.15.32:58242) 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f3318e99700 1 -- 172.21.15.32:0/548018580 learned_addr learned my addr 172.21.15.32:0/548018580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.265+0000 7f331969a700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3314073f90 0x7f33141595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3319e9b700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331415f9e0 0x7f331409cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3318e99700 1 -- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3314073f90 msgr2=0x7f33141595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3318e99700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3314073f90 0x7f33141595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3318e99700 1 -- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331415f9e0 msgr2=0x7f331409cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3318e99700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331415f9e0 0x7f331409cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3318e99700 1 -- 172.21.15.32:0/548018580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040094a0 con 0x7f33140cf610 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3319e9b700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331415f9e0 0x7f331409cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f331969a700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3314073f90 0x7f33141595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3318e99700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 0x7f33140c5bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3310001eb0 tx=0x7f331000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310005d70 con 0x7f33140cf610 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33100070d0 con 0x7f33140cf610 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331409b260 con 0x7f33140cf610 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310007460 con 0x7f33140cf610 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.266+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3314159b30 con 0x7f33140cf610 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.267+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3314068fb0 con 0x7f33140cf610 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.267+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33100075c0 con 0x7f33140cf610 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.268+0000 7f3309ffb700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f32f806aed0 0x7f32f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.268+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3310088860 con 0x7f33140cf610 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.268+0000 7f331969a700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f32f806aed0 0x7f32f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.269+0000 7f331969a700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f32f806aed0 0x7f32f806d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f33140c7500 tx=0x7f3304008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.272+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3310053be0 con 0x7f33140cf610 2023-11-24T07:59:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.476+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f331406c240 con 0x7f33140cf610 2023-11-24T07:59:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.476+0000 7f3309ffb700 1 -- 172.21.15.32:0/548018580 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3310057200 con 0x7f33140cf610 2023-11-24T07:59:31.478 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f32f806aed0 msgr2=0x7f32f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f32f806aed0 0x7f32f806d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f33140c7500 tx=0x7f3304008030 comp rx=0 tx=0).stop 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 msgr2=0x7f33140c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 0x7f33140c5bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3310001eb0 tx=0x7f331000c860 comp rx=0 tx=0).stop 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 shutdown_connections 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f32f806aed0 0x7f32f806d390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331415f9e0 0x7f331409cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3314073f90 0x7f33141595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.478+0000 7f331b8fe700 1 --2- 172.21.15.32:0/548018580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33140cf610 0x7f33140c5bd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.479+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 >> 172.21.15.32:0/548018580 conn(0x7f33140d0bf0 msgr2=0x7f33140b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.479+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 shutdown_connections 2023-11-24T07:59:31.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.479+0000 7f331b8fe700 1 -- 172.21.15.32:0/548018580 wait complete. 2023-11-24T07:59:31.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:59:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.616+0000 7f4f16fe7700 1 Processor -- start 2023-11-24T07:59:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.616+0000 7f4f16fe7700 1 -- start start 2023-11-24T07:59:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.617+0000 7f4f16fe7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4f100b0ba0 0x7f4f100af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.617+0000 7f4f16fe7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f100b0f80 0x7f4f100cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f16fe7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f100b2bd0 con 0x7f4f1015ee90 2023-11-24T07:59:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f16fe7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f100b1fc0 con 0x7f4f100b0f80 2023-11-24T07:59:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f16fe7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f100b22a0 con 0x7f4f100b0ba0 2023-11-24T07:59:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f100b0f80 0x7f4f100cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f100b0f80 0x7f4f100cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58250/0 (socket says 172.21.15.32:58250) 2023-11-24T07:59:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 -- 172.21.15.32:0/4025845756 learned_addr learned my addr 172.21.15.32:0/4025845756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f14d83700 1 --2- 172.21.15.32:0/4025845756 >> v2:172.21.15.123:3300/0 conn(0x7f4f100b0ba0 0x7f4f100af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 -- 172.21.15.32:0/4025845756 >> v2:172.21.15.123:3300/0 conn(0x7f4f100b0ba0 msgr2=0x7f4f100af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 --2- 172.21.15.32:0/4025845756 >> v2:172.21.15.123:3300/0 conn(0x7f4f100b0ba0 0x7f4f100af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 -- 172.21.15.32:0/4025845756 >> v1:172.21.15.123:6789/0 conn(0x7f4f1015ee90 legacy=0x7f4f100ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 -- 172.21.15.32:0/4025845756 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f100b2580 con 0x7f4f100b0f80 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0ffff700 1 --2- 172.21.15.32:0/4025845756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f100b0f80 0x7f4f100cc760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4f100d1340 tx=0x7f4efc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0177b684af39c0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0f7fe700 1 -- 172.21.15.32:0/4025845756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc004230 con 0x7f4f100b0f80 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0f7fe700 1 -- 172.21.15.32:0/4025845756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc004390 con 0x7f4f100b0f80 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.618+0000 7f4f0f7fe700 1 -- 172.21.15.32:0/4025845756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0036a0 con 0x7f4f100b0f80 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 -- 172.21.15.32:0/4025845756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f100b0f80 msgr2=0x7f4f100cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/4025845756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f100b0f80 0x7f4f100cc760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4f100d1340 tx=0x7f4efc009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 -- 172.21.15.32:0/4025845756 shutdown_connections 2023-11-24T07:59:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/4025845756 >> v2:172.21.15.123:3300/0 conn(0x7f4f100b0ba0 0x7f4f100af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/4025845756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f100b0f80 0x7f4f100cc760 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 -- 172.21.15.32:0/4025845756 >> 172.21.15.32:0/4025845756 conn(0x7f4f100b57b0 msgr2=0x7f4f100b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 -- 172.21.15.32:0/4025845756 shutdown_connections 2023-11-24T07:59:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.619+0000 7f4f16fe7700 1 -- 172.21.15.32:0/4025845756 wait complete. 2023-11-24T07:59:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.620+0000 7f4f16fe7700 1 Processor -- start 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.620+0000 7f4f16fe7700 1 -- start start 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f16fe7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f100b0ba0 0x7f4f10157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f16fe7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f100b0f80 0x7f4f1015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f16fe7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 0x7f4f100c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f16fe7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100cb8c0 con 0x7f4f1015ee90 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f16fe7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100ca690 con 0x7f4f100b0f80 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f16fe7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100ca0d0 con 0x7f4f100b0ba0 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f15584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 0x7f4f100c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f15584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 0x7f4f100c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58258/0 (socket says 172.21.15.32:58258) 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f15584700 1 -- 172.21.15.32:0/506392304 learned_addr learned my addr 172.21.15.32:0/506392304 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f14d83700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f100b0ba0 0x7f4f10157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.621+0000 7f4f0ffff700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f100b0f80 0x7f4f1015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f15584700 1 -- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f100b0ba0 msgr2=0x7f4f10157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f15584700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f100b0ba0 0x7f4f10157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f15584700 1 -- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f100b0f80 msgr2=0x7f4f1015b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f15584700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f100b0f80 0x7f4f1015b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f15584700 1 -- 172.21.15.32:0/506392304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc009550 con 0x7f4f1015ee90 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f15584700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 0x7f4f100c9610 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800b970 tx=0x7f4f0800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800d860 con 0x7f4f1015ee90 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0800d9c0 con 0x7f4f1015ee90 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f100ca3b0 con 0x7f4f1015ee90 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800dd70 con 0x7f4f1015ee90 2023-11-24T07:59:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.622+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f100cbdd0 con 0x7f4f1015ee90 2023-11-24T07:59:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.623+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f10067f60 con 0x7f4f1015ee90 2023-11-24T07:59:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.623+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f0800e430 con 0x7f4f1015ee90 2023-11-24T07:59:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.624+0000 7f4f0d7fa700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4ef406aed0 0x7f4ef406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.624+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f08089350 con 0x7f4f1015ee90 2023-11-24T07:59:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.624+0000 7f4f14d83700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4ef406aed0 0x7f4ef406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.625+0000 7f4f14d83700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4ef406aed0 0x7f4ef406d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f04006b20 tx=0x7f4f04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.629+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f080547e0 con 0x7f4f1015ee90 2023-11-24T07:59:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.833+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4f10071f00 con 0x7f4f1015ee90 2023-11-24T07:59:31.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.833+0000 7f4f0d7fa700 1 -- 172.21.15.32:0/506392304 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4f08057e00 con 0x7f4f1015ee90 2023-11-24T07:59:31.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:31.836 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-11-24T07:59:31.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:31.836 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-11-24T07:59:31.836 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-11-24T07:59:31.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:31.836 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-11-24T07:59:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4ef406aed0 msgr2=0x7f4ef406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4ef406aed0 0x7f4ef406d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f04006b20 tx=0x7f4f04008040 comp rx=0 tx=0).stop 2023-11-24T07:59:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 msgr2=0x7f4f100c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 0x7f4f100c9610 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800b970 tx=0x7f4f0800bc80 comp rx=0 tx=0).stop 2023-11-24T07:59:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 shutdown_connections 2023-11-24T07:59:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f4ef406aed0 0x7f4ef406d390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f1015ee90 0x7f4f100c9610 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4f100b0ba0 0x7f4f10157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 --2- 172.21.15.32:0/506392304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f100b0f80 0x7f4f1015b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 >> 172.21.15.32:0/506392304 conn(0x7f4f100b57b0 msgr2=0x7f4f100d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 shutdown_connections 2023-11-24T07:59:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.835+0000 7f4f16fe7700 1 -- 172.21.15.32:0/506392304 wait complete. 2023-11-24T07:59:31.849 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:59:31.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:59:31.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.977+0000 7fd08c4a1700 1 Processor -- start 2023-11-24T07:59:31.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.977+0000 7fd08c4a1700 1 -- start start 2023-11-24T07:59:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08c4a1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd0840b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08c4a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0840b1ba0 con 0x7fd0840cfae0 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08c4a1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0840cff70 con 0x7fd0840b1720 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08c4a1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd084161c80 con 0x7fd08415e020 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08aa3e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0840cfae0 0x7fd0840af3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59978/0 (socket says 172.21.15.32:59978) 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08aa3e700 1 -- 172.21.15.32:0/1822642177 learned_addr learned my addr 172.21.15.32:0/1822642177 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08a23d700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd0840b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.978+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201372962 0 0) 0x7fd0840b1ba0 con 0x7fd0840cfae0 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0700035b0 con 0x7fd0840cfae0 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782928872 0 0) 0x7fd0840cff70 con 0x7fd0840b1720 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0840b1ba0 con 0x7fd0840b1720 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1050056280 0 0) 0x7fd0700035b0 con 0x7fd0840cfae0 2023-11-24T07:59:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 msgr2=0x7fd0840b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd0840b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 >> v1:172.21.15.123:6789/0 conn(0x7fd0840b1720 legacy=0x7fd0840cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084161f40 con 0x7fd0840cfae0 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fd07c0019f0 con 0x7fd0840cfae0 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08a23d700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd0840b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 >> v1:172.21.15.32:6789/0 conn(0x7fd0840cfae0 legacy=0x7fd0840af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070003bc0 0x7fd070003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd070004850 0x7fd070004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd070005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084161c80 con 0x7fd070004850 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07000baf0 con 0x7fd070003bc0 2023-11-24T07:59:31.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07000bcb0 con 0x7fd08415e020 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd089a3c700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd070004850 0x7fd070004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08a23d700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070003bc0 0x7fd070003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd08aa3e700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd070005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd089a3c700 1 -- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 msgr2=0x7fd070005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd089a3c700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08415e020 0x7fd070005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd089a3c700 1 -- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070003bc0 msgr2=0x7fd070003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd089a3c700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070003bc0 0x7fd070003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.979+0000 7fd089a3c700 1 -- 172.21.15.32:0/1822642177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740094a0 con 0x7fd070004850 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd089a3c700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd070004850 0x7fd070004cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd080007fd0 tx=0x7fd070005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a76a9099ed778d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080001430 con 0x7fd070004850 2023-11-24T07:59:31.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0800052d0 con 0x7fd070004850 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08923b700 1 -- 172.21.15.32:0/1822642177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080001d40 con 0x7fd070004850 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08c4a1700 1 -- 172.21.15.32:0/1822642177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd070004850 msgr2=0x7fd070004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/1822642177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd070004850 0x7fd070004cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd080007fd0 tx=0x7fd070005880 comp rx=0 tx=0).stop 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08aa3e700 1 -- 172.21.15.32:0/1822642177 reap_dead start 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08c4a1700 1 -- 172.21.15.32:0/1822642177 shutdown_connections 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08c4a1700 1 -- 172.21.15.32:0/1822642177 >> 172.21.15.32:0/1822642177 conn(0x7fd084072710 msgr2=0x7fd084072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08c4a1700 1 -- 172.21.15.32:0/1822642177 shutdown_connections 2023-11-24T07:59:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.980+0000 7fd08c4a1700 1 -- 172.21.15.32:0/1822642177 wait complete. 2023-11-24T07:59:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 Processor -- start 2023-11-24T07:59:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 -- start start 2023-11-24T07:59:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070004850 0x7fd084098f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0840996b0 0x7fd0841f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 0x7fd0841f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0841f4a20 con 0x7fd0840625d0 2023-11-24T07:59:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0841f4ca0 con 0x7fd070004850 2023-11-24T07:59:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08c4a1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd084155b80 con 0x7fd0840996b0 2023-11-24T07:59:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08aa3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 0x7fd0841f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08aa3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 0x7fd0841f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58278/0 (socket says 172.21.15.32:58278) 2023-11-24T07:59:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08aa3e700 1 -- 172.21.15.32:0/186150845 learned_addr learned my addr 172.21.15.32:0/186150845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd08a23d700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070004850 0x7fd084098f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.982+0000 7fd089a3c700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0840996b0 0x7fd0841f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08aa3e700 1 -- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0840996b0 msgr2=0x7fd0841f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08aa3e700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0840996b0 0x7fd0841f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08aa3e700 1 -- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070004850 msgr2=0x7fd084098f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08aa3e700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070004850 0x7fd084098f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08aa3e700 1 -- 172.21.15.32:0/186150845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0841590e0 con 0x7fd0840625d0 2023-11-24T07:59:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08aa3e700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 0x7fd0841f44e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd0840cdd00 tx=0x7fd07c00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00abf0 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c00ad50 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0841f50f0 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c013630 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.983+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0841593c0 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.984+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0841f4f40 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.985+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd07c013790 con 0x7fd0840625d0 2023-11-24T07:59:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.985+0000 7fd07affd700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06806aed0 0x7fd06806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.985+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd07c08a2a0 con 0x7fd0840625d0 2023-11-24T07:59:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.986+0000 7fd08a23d700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06806aed0 0x7fd06806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.986+0000 7fd08a23d700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06806aed0 0x7fd06806d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd074009d00 tx=0x7fd074009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:31.989+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd07c0557c0 con 0x7fd0840625d0 2023-11-24T07:59:32.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.190+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd0841559e0 con 0x7fd0840625d0 2023-11-24T07:59:32.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.672+0000 7fd07affd700 1 -- 172.21.15.32:0/186150845 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd0841559e0 con 0x7fd0840625d0 2023-11-24T07:59:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06806aed0 msgr2=0x7fd06806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06806aed0 0x7fd06806d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd074009d00 tx=0x7fd074009210 comp rx=0 tx=0).stop 2023-11-24T07:59:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 msgr2=0x7fd0841f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 0x7fd0841f44e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd0840cdd00 tx=0x7fd07c00a5b0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 shutdown_connections 2023-11-24T07:59:32.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd06806aed0 0x7fd06806d390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0840625d0 0x7fd0841f44e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd070004850 0x7fd084098f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 --2- 172.21.15.32:0/186150845 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd0840996b0 0x7fd0841f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 >> 172.21.15.32:0/186150845 conn(0x7fd084072710 msgr2=0x7fd08406ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 shutdown_connections 2023-11-24T07:59:32.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.675+0000 7fd08c4a1700 1 -- 172.21.15.32:0/186150845 wait complete. 2023-11-24T07:59:32.679 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-11-24T07:59:32.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:59:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.826+0000 7fa2dc728700 1 Processor -- start 2023-11-24T07:59:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.826+0000 7fa2dc728700 1 -- start start 2023-11-24T07:59:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2dc728700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2d41587f0 0x7fa2d406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2dc728700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2d415b370 0x7fa2d415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2dc728700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d406b530 con 0x7fa2d40b3c60 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2dc728700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d40ae5e0 con 0x7fa2d415b370 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2dc728700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d415be30 con 0x7fa2d41587f0 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2d9cc3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2d415b370 0x7fa2d415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2d9cc3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2d415b370 0x7fa2d415b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58284/0 (socket says 172.21.15.32:58284) 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2d9cc3700 1 -- 172.21.15.32:0/445700606 learned_addr learned my addr 172.21.15.32:0/445700606 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.827+0000 7fa2da4c4700 1 --2- 172.21.15.32:0/445700606 >> v2:172.21.15.123:3300/0 conn(0x7fa2d41587f0 0x7fa2d406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d9cc3700 1 -- 172.21.15.32:0/445700606 >> v2:172.21.15.123:3300/0 conn(0x7fa2d41587f0 msgr2=0x7fa2d406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d9cc3700 1 --2- 172.21.15.32:0/445700606 >> v2:172.21.15.123:3300/0 conn(0x7fa2d41587f0 0x7fa2d406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d9cc3700 1 -- 172.21.15.32:0/445700606 >> v1:172.21.15.123:6789/0 conn(0x7fa2d40b3c60 legacy=0x7fa2d40af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d9cc3700 1 -- 172.21.15.32:0/445700606 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d415c110 con 0x7fa2d415b370 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d9cc3700 1 --2- 172.21.15.32:0/445700606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d415b800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00098f0 tx=0x7fa2d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160464291c864716 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d94c2700 1 -- 172.21.15.32:0/445700606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000b860 con 0x7fa2d415b370 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d94c2700 1 -- 172.21.15.32:0/445700606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000b9c0 con 0x7fa2d415b370 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2d94c2700 1 -- 172.21.15.32:0/445700606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000bd50 con 0x7fa2d415b370 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2dc728700 1 -- 172.21.15.32:0/445700606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 msgr2=0x7fa2d415b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.828+0000 7fa2dc728700 1 --2- 172.21.15.32:0/445700606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d415b800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00098f0 tx=0x7fa2d0009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.829+0000 7fa2dc728700 1 -- 172.21.15.32:0/445700606 shutdown_connections 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.829+0000 7fa2dc728700 1 --2- 172.21.15.32:0/445700606 >> v2:172.21.15.123:3300/0 conn(0x7fa2d41587f0 0x7fa2d406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.829+0000 7fa2dc728700 1 --2- 172.21.15.32:0/445700606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d415b800 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.829+0000 7fa2dc728700 1 -- 172.21.15.32:0/445700606 >> 172.21.15.32:0/445700606 conn(0x7fa2d40d0bf0 msgr2=0x7fa2d40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.829+0000 7fa2dc728700 1 -- 172.21.15.32:0/445700606 shutdown_connections 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.829+0000 7fa2dc728700 1 -- 172.21.15.32:0/445700606 wait complete. 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 Processor -- start 2023-11-24T07:59:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 -- start start 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40b3c60 0x7fa2d4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2d41587f0 0x7fa2d40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d40afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4150b80 con 0x7fa2d415b370 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4150e60 con 0x7fa2d40b3c60 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.830+0000 7fa2dc728700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4151140 con 0x7fa2d41587f0 2023-11-24T07:59:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d40afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d40afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58292/0 (socket says 172.21.15.32:58292) 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 -- 172.21.15.32:0/999695786 learned_addr learned my addr 172.21.15.32:0/999695786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2da4c4700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40b3c60 0x7fa2d4074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2d9cc3700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2d41587f0 0x7fa2d40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 -- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2d41587f0 msgr2=0x7fa2d40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2d41587f0 0x7fa2d40b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 -- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40b3c60 msgr2=0x7fa2d4074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40b3c60 0x7fa2d4074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 -- 172.21.15.32:0/999695786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009550 con 0x7fa2d415b370 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dacc5700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d40afe40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00cc50 tx=0x7fa2cc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003c00 con 0x7fa2d415b370 2023-11-24T07:59:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc003d60 con 0x7fa2d415b370 2023-11-24T07:59:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d4151420 con 0x7fa2d415b370 2023-11-24T07:59:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00aab0 con 0x7fa2d415b370 2023-11-24T07:59:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.831+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d4074ab0 con 0x7fa2d415b370 2023-11-24T07:59:32.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.832+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d4151a70 con 0x7fa2d415b370 2023-11-24T07:59:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.833+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2cc00ac10 con 0x7fa2d415b370 2023-11-24T07:59:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.833+0000 7fa2caffd700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa2b806aed0 0x7fa2b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.833+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa2cc0892e0 con 0x7fa2d415b370 2023-11-24T07:59:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.834+0000 7fa2da4c4700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa2b806aed0 0x7fa2b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.834+0000 7fa2da4c4700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa2b806aed0 0x7fa2b806d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2d415ba30 tx=0x7fa2c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:32.838+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2cc0544e0 con 0x7fa2d415b370 2023-11-24T07:59:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.036+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa2d4074ab0 con 0x7fa2d415b370 2023-11-24T07:59:33.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.678+0000 7fa2caffd700 1 -- 172.21.15.32:0/999695786 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa2d4074ab0 con 0x7fa2d415b370 2023-11-24T07:59:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa2b806aed0 msgr2=0x7fa2b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa2b806aed0 0x7fa2b806d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2d415ba30 tx=0x7fa2c4008040 comp rx=0 tx=0).stop 2023-11-24T07:59:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 msgr2=0x7fa2d40afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d40afe40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00cc50 tx=0x7fa2cc00cf60 comp rx=0 tx=0).stop 2023-11-24T07:59:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 shutdown_connections 2023-11-24T07:59:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa2b806aed0 0x7fa2b806d390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d415b370 0x7fa2d40afe40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2d40b3c60 0x7fa2d4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 --2- 172.21.15.32:0/999695786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa2d41587f0 0x7fa2d40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.683+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 >> 172.21.15.32:0/999695786 conn(0x7fa2d40d0bf0 msgr2=0x7fa2d4072220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.685+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 shutdown_connections 2023-11-24T07:59:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.685+0000 7fa2dc728700 1 -- 172.21.15.32:0/999695786 wait complete. 2023-11-24T07:59:33.699 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-11-24T07:59:33.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:59:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.842+0000 7f3949258700 1 Processor -- start 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.842+0000 7f3949258700 1 -- start start 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f3949258700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 0x7f39440ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f3949258700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394406e400 con 0x7f39440ad870 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f3949258700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394406e630 con 0x7f39440aedc0 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f3949258700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394406fde0 con 0x7f394415e290 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f394359e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39440ad870 0x7f394406f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60012/0 (socket says 172.21.15.32:60012) 2023-11-24T07:59:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f394359e700 1 -- 172.21.15.32:0/1740266028 learned_addr learned my addr 172.21.15.32:0/1740266028 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f394259c700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 0x7f39440ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477949300 0 0) 0x7f394406e400 con 0x7f39440ad870 2023-11-24T07:59:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.843+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f392c0035b0 con 0x7f39440ad870 2023-11-24T07:59:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1463816941 0 0) 0x7f392c0035b0 con 0x7f39440ad870 2023-11-24T07:59:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 msgr2=0x7f39440ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 0x7f39440ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 >> v1:172.21.15.123:6789/0 conn(0x7f39440aedc0 legacy=0x7f39440af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394406e910 con 0x7f39440ad870 2023-11-24T07:59:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f393c0019f0 con 0x7f39440ad870 2023-11-24T07:59:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 >> v1:172.21.15.32:6789/0 conn(0x7f39440ad870 legacy=0x7f394406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f392c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c0043e0 0x7f392c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 0x7f392c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394406fde0 con 0x7f392c003a30 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00baf0 con 0x7f392c0043e0 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00bcb0 con 0x7f394415e290 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f392c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f394259c700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c0043e0 0x7f392c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f394359e700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 0x7f392c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 -- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 msgr2=0x7f392c0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415e290 0x7f392c0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 -- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c0043e0 msgr2=0x7f392c0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c0043e0 0x7f392c0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 -- 172.21.15.32:0/1740266028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0035b0 con 0x7f392c003a30 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3942d9d700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f392c003e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39380018d0 tx=0x7f392c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669d15330b42eb3e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800c690 con 0x7f392c003a30 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393800c810 con 0x7f392c003a30 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.844+0000 7f3941d9b700 1 -- 172.21.15.32:0/1740266028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800cbc0 con 0x7f392c003a30 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f3949258700 1 -- 172.21.15.32:0/1740266028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 msgr2=0x7f392c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f3949258700 1 --2- 172.21.15.32:0/1740266028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f392c003e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39380018d0 tx=0x7f392c005880 comp rx=0 tx=0).stop 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f394359e700 1 -- 172.21.15.32:0/1740266028 reap_dead start 2023-11-24T07:59:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f3949258700 1 -- 172.21.15.32:0/1740266028 shutdown_connections 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f3949258700 1 -- 172.21.15.32:0/1740266028 >> 172.21.15.32:0/1740266028 conn(0x7f39440b2050 msgr2=0x7f39440b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f3949258700 1 -- 172.21.15.32:0/1740266028 shutdown_connections 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.845+0000 7f3949258700 1 -- 172.21.15.32:0/1740266028 wait complete. 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 Processor -- start 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 -- start start 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f39441422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f394415fb00 0x7f394415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415b580 0x7f394415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441604c0 con 0x7f392c003a30 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944143280 con 0x7f394415fb00 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3949258700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3944142c70 con 0x7f394415b580 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3942d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f39441422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3942d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f39441422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58308/0 (socket says 172.21.15.32:58308) 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.847+0000 7f3942d9d700 1 -- 172.21.15.32:0/458565517 learned_addr learned my addr 172.21.15.32:0/458565517 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f394359e700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415b580 0x7f394415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f394259c700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f394415fb00 0x7f394415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3942d9d700 1 -- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415b580 msgr2=0x7f394415ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3942d9d700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415b580 0x7f394415ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3942d9d700 1 -- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f394415fb00 msgr2=0x7f394415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3942d9d700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f394415fb00 0x7f394415ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3942d9d700 1 -- 172.21.15.32:0/458565517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39441449c0 con 0x7f392c003a30 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f394259c700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f394415fb00 0x7f394415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3942d9d700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f39441422c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f393800c0e0 tx=0x7f3938001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800cb80 con 0x7f392c003a30 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938021070 con 0x7f392c003a30 2023-11-24T07:59:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394415c2d0 con 0x7f392c003a30 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800cee0 con 0x7f392c003a30 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.848+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394415c5b0 con 0x7f392c003a30 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.849+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3944068d90 con 0x7f392c003a30 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.850+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3938023030 con 0x7f392c003a30 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.850+0000 7f392b7fe700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f393006aed0 0x7f393006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.850+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f393808a700 con 0x7f392c003a30 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.850+0000 7f394259c700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f393006aed0 0x7f393006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.851+0000 7f394259c700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f393006aed0 0x7f393006d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f394415d4c0 tx=0x7f3934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:33.855+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3938055960 con 0x7f392c003a30 2023-11-24T07:59:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.104+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f394415a000 con 0x7f392c003a30 2023-11-24T07:59:34.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.693+0000 7f392b7fe700 1 -- 172.21.15.32:0/458565517 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3938058f80 con 0x7f392c003a30 2023-11-24T07:59:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.716+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f393006aed0 msgr2=0x7f393006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.716+0000 7f3949258700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f393006aed0 0x7f393006d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f394415d4c0 tx=0x7f3934008040 comp rx=0 tx=0).stop 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.716+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 msgr2=0x7f39441422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.716+0000 7f3949258700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f39441422c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f393800c0e0 tx=0x7f3938001230 comp rx=0 tx=0).stop 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.717+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 shutdown_connections 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.717+0000 7f3949258700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f393006aed0 0x7f393006d390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.717+0000 7f3949258700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f394415b580 0x7f394415ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.717+0000 7f3949258700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f392c003a30 0x7f39441422c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.717+0000 7f3949258700 1 --2- 172.21.15.32:0/458565517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f394415fb00 0x7f394415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.717+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 >> 172.21.15.32:0/458565517 conn(0x7f39440b2050 msgr2=0x7f39440b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.718+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 shutdown_connections 2023-11-24T07:59:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.718+0000 7f3949258700 1 -- 172.21.15.32:0/458565517 wait complete. 2023-11-24T07:59:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T07:59:34.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.862+0000 7fec25890700 1 Processor -- start 2023-11-24T07:59:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.862+0000 7fec25890700 1 -- start start 2023-11-24T07:59:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec25890700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec2015e1e0 0x7fec200d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec25890700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200d20b0 con 0x7fec200d0f80 2023-11-24T07:59:34.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec25890700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200d2390 con 0x7fec200d3a00 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec25890700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec200d3150 con 0x7fec2015e1e0 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec1f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec200d0f80 0x7fec200b6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60046/0 (socket says 172.21.15.32:60046) 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec1f7fe700 1 -- 172.21.15.32:0/331170144 learned_addr learned my addr 172.21.15.32:0/331170144 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.863+0000 7fec1effd700 1 --2- 172.21.15.32:0/331170144 >> v2:172.21.15.123:3300/0 conn(0x7fec2015e1e0 0x7fec200d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356529781 0 0) 0x7fec200d20b0 con 0x7fec200d0f80 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec080035b0 con 0x7fec200d0f80 2023-11-24T07:59:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2012788853 0 0) 0x7fec080035b0 con 0x7fec200d0f80 2023-11-24T07:59:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 >> v2:172.21.15.123:3300/0 conn(0x7fec2015e1e0 msgr2=0x7fec200d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 --2- 172.21.15.32:0/331170144 >> v2:172.21.15.123:3300/0 conn(0x7fec2015e1e0 0x7fec200d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 >> v1:172.21.15.123:6789/0 conn(0x7fec200d3a00 legacy=0x7fec200d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200d2670 con 0x7fec200d0f80 2023-11-24T07:59:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fec180019f0 con 0x7fec200d0f80 2023-11-24T07:59:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1effd700 1 --2- 172.21.15.32:0/331170144 >> v2:172.21.15.123:3300/0 conn(0x7fec2015e1e0 0x7fec200d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 >> v1:172.21.15.32:6789/0 conn(0x7fec200d0f80 legacy=0x7fec200b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08003a00 0x7fec08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec08004400 0x7fec080048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec080050d0 0x7fec0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d3150 con 0x7fec08004400 2023-11-24T07:59:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0800bf80 con 0x7fec080050d0 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0800c140 con 0x7fec08003a00 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec08004400 0x7fec080048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1f7fe700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec080050d0 0x7fec0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1effd700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08003a00 0x7fec08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1e7fc700 1 -- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08003a00 msgr2=0x7fec08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08003a00 0x7fec08003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1e7fc700 1 -- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec080050d0 msgr2=0x7fec0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec080050d0 0x7fec0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1e7fc700 1 -- 172.21.15.32:0/331170144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec080035b0 con 0x7fec08004400 2023-11-24T07:59:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.864+0000 7fec1f7fe700 1 -- 172.21.15.32:0/331170144 reap_dead start 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec08004400 0x7fec080048d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fec200d3e30 tx=0x7fec08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f51c6ffe678e7c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec100052d0 con 0x7fec08004400 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10005430 con 0x7fec08004400 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec1dffb700 1 -- 172.21.15.32:0/331170144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec100038a0 con 0x7fec08004400 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 -- 172.21.15.32:0/331170144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec08004400 msgr2=0x7fec080048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec08004400 0x7fec080048d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fec200d3e30 tx=0x7fec08005880 comp rx=0 tx=0).stop 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 -- 172.21.15.32:0/331170144 shutdown_connections 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 --2- 172.21.15.32:0/331170144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec08004400 0x7fec080048d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 -- 172.21.15.32:0/331170144 >> 172.21.15.32:0/331170144 conn(0x7fec200b5b50 msgr2=0x7fec200b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 -- 172.21.15.32:0/331170144 shutdown_connections 2023-11-24T07:59:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.865+0000 7fec25890700 1 -- 172.21.15.32:0/331170144 wait complete. 2023-11-24T07:59:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.867+0000 7fec25890700 1 Processor -- start 2023-11-24T07:59:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.867+0000 7fec25890700 1 -- start start 2023-11-24T07:59:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08004400 0x7fec2015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 0x7fec2015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec2015bcd0 0x7fec20140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200cafd0 con 0x7fec2015aab0 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200cb2b0 con 0x7fec2015bcd0 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec200cb590 con 0x7fec08004400 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 0x7fec2015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 0x7fec2015b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58344/0 (socket says 172.21.15.32:58344) 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 -- 172.21.15.32:0/3345881852 learned_addr learned my addr 172.21.15.32:0/3345881852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1effd700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08004400 0x7fec2015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1f7fe700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec2015bcd0 0x7fec20140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 -- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08004400 msgr2=0x7fec2015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08004400 0x7fec2015eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 -- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec2015bcd0 msgr2=0x7fec20140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec2015bcd0 0x7fec20140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 -- 172.21.15.32:0/3345881852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2015b270 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1f7fe700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec2015bcd0 0x7fec20140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec1e7fc700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 0x7fec2015b790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec0800bf50 tx=0x7fec10001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1001e630 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1001e7b0 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2015af30 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1001eba0 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.868+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec20140790 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.869+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2015b400 con 0x7fec2015aab0 2023-11-24T07:59:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.870+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec1001ed00 con 0x7fec2015aab0 2023-11-24T07:59:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.870+0000 7fec07fff700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fec0c06aed0 0x7fec0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.870+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec1008a4b0 con 0x7fec2015aab0 2023-11-24T07:59:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.871+0000 7fec1effd700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fec0c06aed0 0x7fec0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.871+0000 7fec1effd700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fec0c06aed0 0x7fec0c06d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec14009c10 tx=0x7fec14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:34.875+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec10055570 con 0x7fec2015aab0 2023-11-24T07:59:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.076+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec200751e0 con 0x7fec2015aab0 2023-11-24T07:59:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.077+0000 7fec07fff700 1 -- 172.21.15.32:0/3345881852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16570 (secure 0 0 0) 0x7fec10058b90 con 0x7fec2015aab0 2023-11-24T07:59:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:35.079 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:34.105685+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:35.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fec0c06aed0 msgr2=0x7fec0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fec0c06aed0 0x7fec0c06d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fec14009c10 tx=0x7fec14008040 comp rx=0 tx=0).stop 2023-11-24T07:59:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 msgr2=0x7fec2015b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 0x7fec2015b790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec0800bf50 tx=0x7fec10001230 comp rx=0 tx=0).stop 2023-11-24T07:59:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 shutdown_connections 2023-11-24T07:59:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fec0c06aed0 0x7fec0c06d390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec2015bcd0 0x7fec20140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fec08004400 0x7fec2015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 --2- 172.21.15.32:0/3345881852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2015aab0 0x7fec2015b790 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 >> 172.21.15.32:0/3345881852 conn(0x7fec200b5b50 msgr2=0x7fec201531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 shutdown_connections 2023-11-24T07:59:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.079+0000 7fec25890700 1 -- 172.21.15.32:0/3345881852 wait complete. 2023-11-24T07:59:35.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:35.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.217+0000 7f6af05d6700 1 Processor -- start 2023-11-24T07:59:35.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.217+0000 7f6af05d6700 1 -- start start 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6af05d6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ae80af4d0 0x7f6ae80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6af05d6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae80ae430 con 0x7f6ae806c710 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6af05d6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae80ae710 con 0x7f6ae815e080 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6af05d6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae80af0b0 con 0x7f6ae80af4d0 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aedb71700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ae806c710 0x7f6ae806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60062/0 (socket says 172.21.15.32:60062) 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aedb71700 1 -- 172.21.15.32:0/3267902727 learned_addr learned my addr 172.21.15.32:0/3267902727 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:35.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aeeb73700 1 --2- 172.21.15.32:0/3267902727 >> v2:172.21.15.123:3300/0 conn(0x7f6ae80af4d0 0x7f6ae80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002375969 0 0) 0x7f6ae80ae430 con 0x7f6ae806c710 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad40035b0 con 0x7f6ae806c710 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070639476 0 0) 0x7f6ad40035b0 con 0x7f6ae806c710 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 >> v2:172.21.15.123:3300/0 conn(0x7f6ae80af4d0 msgr2=0x7f6ae80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 --2- 172.21.15.32:0/3267902727 >> v2:172.21.15.123:3300/0 conn(0x7f6ae80af4d0 0x7f6ae80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 >> v1:172.21.15.123:6789/0 conn(0x7f6ae815e080 legacy=0x7f6ae806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.218+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80ae9f0 con 0x7f6ae806c710 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f6ad8001670 con 0x7f6ae806c710 2023-11-24T07:59:35.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 >> v1:172.21.15.32:6789/0 conn(0x7f6ae806c710 legacy=0x7f6ae806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ad4003a00 0x7f6ad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ad40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ad40050d0 0x7f6ad400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80af0b0 con 0x7f6ad4004400 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad400c030 con 0x7f6ad40050d0 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad400c1f0 con 0x7f6ad4003a00 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aeeb73700 1 --2- 172.21.15.32:0/3267902727 >> v2:172.21.15.123:3300/0 conn(0x7f6ae80af4d0 0x7f6ae80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ad40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aee372700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ad4003a00 0x7f6ad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aeeb73700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ad40050d0 0x7f6ad400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 -- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ad4003a00 msgr2=0x7f6ad4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ad4003a00 0x7f6ad4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 -- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ad40050d0 msgr2=0x7f6ad400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ad40050d0 0x7f6ad400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 -- 172.21.15.32:0/3267902727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40035b0 con 0x7f6ad4004400 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aeeb73700 1 -- 172.21.15.32:0/3267902727 reap_dead start 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aedb71700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ad40048d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c2f0 tx=0x7f6ad4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657409b7b48879b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800ccf0 con 0x7f6ad4004400 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad800ce70 con 0x7f6ad4004400 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.219+0000 7f6aed370700 1 -- 172.21.15.32:0/3267902727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8005b80 con 0x7f6ad4004400 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 -- 172.21.15.32:0/3267902727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 msgr2=0x7f6ad40048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ad40048d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c2f0 tx=0x7f6ad4005880 comp rx=0 tx=0).stop 2023-11-24T07:59:35.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 -- 172.21.15.32:0/3267902727 shutdown_connections 2023-11-24T07:59:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 --2- 172.21.15.32:0/3267902727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ad40048d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 -- 172.21.15.32:0/3267902727 >> 172.21.15.32:0/3267902727 conn(0x7f6ae8071b90 msgr2=0x7f6ae8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 -- 172.21.15.32:0/3267902727 shutdown_connections 2023-11-24T07:59:35.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.220+0000 7f6af05d6700 1 -- 172.21.15.32:0/3267902727 wait complete. 2023-11-24T07:59:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.221+0000 7f6af05d6700 1 Processor -- start 2023-11-24T07:59:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.221+0000 7f6af05d6700 1 -- start start 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6af05d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ae8153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6af05d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8156fa0 0x7f6ae815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6af05d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae81548a0 0x7f6ae815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6af05d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81540d0 con 0x7f6ad4004400 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6af05d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8160090 con 0x7f6ae8156fa0 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6af05d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80d0770 con 0x7f6ae81548a0 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ae8153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ae8153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58362/0 (socket says 172.21.15.32:58362) 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 -- 172.21.15.32:0/2767409172 learned_addr learned my addr 172.21.15.32:0/2767409172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aedb71700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8156fa0 0x7f6ae815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aeeb73700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae81548a0 0x7f6ae815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 -- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae81548a0 msgr2=0x7f6ae815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae81548a0 0x7f6ae815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 -- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8156fa0 msgr2=0x7f6ae815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8156fa0 0x7f6ae815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.222+0000 7f6aee372700 1 -- 172.21.15.32:0/2767409172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae815f760 con 0x7f6ad4004400 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.223+0000 7f6aee372700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ae8153b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400f7b0 tx=0x7f6ae400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.223+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40052e0 con 0x7f6ad4004400 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.223+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae4005460 con 0x7f6ad4004400 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.223+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae815fa40 con 0x7f6ad4004400 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.223+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40038c0 con 0x7f6ad4004400 2023-11-24T07:59:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.223+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae8154270 con 0x7f6ad4004400 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.224+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae80753f0 con 0x7f6ad4004400 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.224+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ae4003a20 con 0x7f6ad4004400 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.225+0000 7f6adeffd700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6acc06aed0 0x7f6acc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.225+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ae40893a0 con 0x7f6ad4004400 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.225+0000 7f6aedb71700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6acc06aed0 0x7f6acc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.226+0000 7f6aedb71700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6acc06aed0 0x7f6acc06d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80b1c20 tx=0x7f6ad8007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.229+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ae40544e0 con 0x7f6ad4004400 2023-11-24T07:59:35.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.430+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ae8067f60 con 0x7f6ad4004400 2023-11-24T07:59:35.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.431+0000 7f6adeffd700 1 -- 172.21.15.32:0/2767409172 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f6ae4057b00 con 0x7f6ad4004400 2023-11-24T07:59:35.433 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:35.433 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:34.105685+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:35.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6acc06aed0 msgr2=0x7f6acc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6acc06aed0 0x7f6acc06d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80b1c20 tx=0x7f6ad8007000 comp rx=0 tx=0).stop 2023-11-24T07:59:35.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 msgr2=0x7f6ae8153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ae8153b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400f7b0 tx=0x7f6ae400fb70 comp rx=0 tx=0).stop 2023-11-24T07:59:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 shutdown_connections 2023-11-24T07:59:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6acc06aed0 0x7f6acc06d390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae81548a0 0x7f6ae815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4004400 0x7f6ae8153b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 --2- 172.21.15.32:0/2767409172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8156fa0 0x7f6ae815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.433+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 >> 172.21.15.32:0/2767409172 conn(0x7f6ae8071b90 msgr2=0x7f6ae8156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.434+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 shutdown_connections 2023-11-24T07:59:35.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.434+0000 7f6af05d6700 1 -- 172.21.15.32:0/2767409172 wait complete. 2023-11-24T07:59:35.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:59:35.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.575+0000 7f054b76a700 1 Processor -- start 2023-11-24T07:59:35.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.575+0000 7f054b76a700 1 -- start start 2023-11-24T07:59:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f054b76a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05440746a0 0x7f05440b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f054b76a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f054406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f054b76a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05440b3c60 con 0x7f054415e160 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f054b76a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05440b4500 con 0x7f05440746a0 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f054b76a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b58c0 con 0x7f05440afc90 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05440746a0 0x7f05440b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05440746a0 0x7f05440b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58372/0 (socket says 172.21.15.32:58372) 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 -- 172.21.15.32:0/459798901 learned_addr learned my addr 172.21.15.32:0/459798901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0549d07700 1 --2- 172.21.15.32:0/459798901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f054406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 -- 172.21.15.32:0/459798901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 msgr2=0x7f054406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 --2- 172.21.15.32:0/459798901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f054406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 -- 172.21.15.32:0/459798901 >> v1:172.21.15.123:6789/0 conn(0x7f054415e160 legacy=0x7f0544073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 -- 172.21.15.32:0/459798901 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440b0120 con 0x7f05440746a0 2023-11-24T07:59:35.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.576+0000 7f0548d05700 1 --2- 172.21.15.32:0/459798901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05440746a0 0x7f05440b5da0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f05340098f0 tx=0x7f0534009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42d288030cb433f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f053bfff700 1 -- 172.21.15.32:0/459798901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534011830 con 0x7f05440746a0 2023-11-24T07:59:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f053bfff700 1 -- 172.21.15.32:0/459798901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534011990 con 0x7f05440746a0 2023-11-24T07:59:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f053bfff700 1 -- 172.21.15.32:0/459798901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534011d20 con 0x7f05440746a0 2023-11-24T07:59:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 -- 172.21.15.32:0/459798901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05440746a0 msgr2=0x7f05440b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 --2- 172.21.15.32:0/459798901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05440746a0 0x7f05440b5da0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f05340098f0 tx=0x7f0534009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 -- 172.21.15.32:0/459798901 shutdown_connections 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 --2- 172.21.15.32:0/459798901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f054406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 --2- 172.21.15.32:0/459798901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05440746a0 0x7f05440b5da0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 -- 172.21.15.32:0/459798901 >> 172.21.15.32:0/459798901 conn(0x7f05440d0bf0 msgr2=0x7f05440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 -- 172.21.15.32:0/459798901 shutdown_connections 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.578+0000 7f054b76a700 1 -- 172.21.15.32:0/459798901 wait complete. 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.579+0000 7f054b76a700 1 Processor -- start 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 -- start start 2023-11-24T07:59:35.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440746a0 0x7f0544160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f0544148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 0x7f0544155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440725e0 con 0x7f054415e160 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05441608e0 con 0x7f05440746a0 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f054b76a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05441612b0 con 0x7f05440afc90 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f0549d07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 0x7f0544155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.580+0000 7f0549d07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 0x7f0544155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58380/0 (socket says 172.21.15.32:58380) 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 -- 172.21.15.32:0/2986037102 learned_addr learned my addr 172.21.15.32:0/2986037102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0548d05700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f0544148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549506700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440746a0 0x7f0544160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 -- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 msgr2=0x7f0544148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f0544148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 -- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440746a0 msgr2=0x7f0544160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440746a0 0x7f0544160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 -- 172.21.15.32:0/2986037102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534009550 con 0x7f054415e160 2023-11-24T07:59:35.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0549d07700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 0x7f0544155000 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f053c009dd0 tx=0x7f053c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c009c40 con 0x7f054415e160 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c00f040 con 0x7f054415e160 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0544072b10 con 0x7f054415e160 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00ceb0 con 0x7f054415e160 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.581+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0544072830 con 0x7f054415e160 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.582+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0544062580 con 0x7f054415e160 2023-11-24T07:59:35.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.583+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f053c014480 con 0x7f054415e160 2023-11-24T07:59:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.583+0000 7f0539ffb700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f052806aed0 0x7f052806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.584+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f053c089ea0 con 0x7f054415e160 2023-11-24T07:59:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.584+0000 7f0549506700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f052806aed0 0x7f052806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.584+0000 7f0549506700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f052806aed0 0x7f052806d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0540007fd0 tx=0x7f0540009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.588+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f053c054eb0 con 0x7f054415e160 2023-11-24T07:59:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:35.839+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05440b6160 con 0x7f054415e160 2023-11-24T07:59:36.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.700+0000 7f0539ffb700 1 -- 172.21.15.32:0/2986037102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f053c0180b0 con 0x7f054415e160 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f052806aed0 msgr2=0x7f052806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f052806aed0 0x7f052806d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0540007fd0 tx=0x7f0540009040 comp rx=0 tx=0).stop 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 msgr2=0x7f0544155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 0x7f0544155000 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f053c009dd0 tx=0x7f053c00c3a0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 shutdown_connections 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f052806aed0 0x7f052806d390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f054415e160 0x7f0544155000 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05440746a0 0x7f0544160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 --2- 172.21.15.32:0/2986037102 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05440afc90 0x7f0544148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 >> 172.21.15.32:0/2986037102 conn(0x7f05440d0bf0 msgr2=0x7f054415c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 shutdown_connections 2023-11-24T07:59:36.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.703+0000 7f054b76a700 1 -- 172.21.15.32:0/2986037102 wait complete. 2023-11-24T07:59:36.719 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:59:36.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:59:36.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.853+0000 7fa59f0cc700 1 Processor -- start 2023-11-24T07:59:36.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.853+0000 7fa59f0cc700 1 -- start start 2023-11-24T07:59:36.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa59f0cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5980c6c30 0x7fa5980cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa59f0cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa5980cef40 0x7fa5980cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa59f0cc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5980d0140 con 0x7fa59815bf50 2023-11-24T07:59:36.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa59f0cc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59815c330 con 0x7fa5980c6c30 2023-11-24T07:59:36.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa59f0cc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5980cf3d0 con 0x7fa5980cef40 2023-11-24T07:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa597fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5980c6c30 0x7fa5980cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa597fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5980c6c30 0x7fa5980cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58386/0 (socket says 172.21.15.32:58386) 2023-11-24T07:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.854+0000 7fa597fff700 1 -- 172.21.15.32:0/4231617135 learned_addr learned my addr 172.21.15.32:0/4231617135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59d669700 1 --2- 172.21.15.32:0/4231617135 >> v2:172.21.15.123:3300/0 conn(0x7fa5980cef40 0x7fa5980cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa597fff700 1 -- 172.21.15.32:0/4231617135 >> v2:172.21.15.123:3300/0 conn(0x7fa5980cef40 msgr2=0x7fa5980cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa597fff700 1 --2- 172.21.15.32:0/4231617135 >> v2:172.21.15.123:3300/0 conn(0x7fa5980cef40 0x7fa5980cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa597fff700 1 -- 172.21.15.32:0/4231617135 >> v1:172.21.15.32:6789/0 conn(0x7fa59815bf50 legacy=0x7fa5980ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa597fff700 1 -- 172.21.15.32:0/4231617135 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980d03e0 con 0x7fa5980c6c30 2023-11-24T07:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59d669700 1 --2- 172.21.15.32:0/4231617135 >> v2:172.21.15.123:3300/0 conn(0x7fa5980cef40 0x7fa5980cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa597fff700 1 --2- 172.21.15.32:0/4231617135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5980cea00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5980c8c00 tx=0x7fa584009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9710c49f3f78db1b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa5977fe700 1 -- 172.21.15.32:0/4231617135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400b870 con 0x7fa5980c6c30 2023-11-24T07:59:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa5977fe700 1 -- 172.21.15.32:0/4231617135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58400b9d0 con 0x7fa5980c6c30 2023-11-24T07:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa5977fe700 1 -- 172.21.15.32:0/4231617135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400bd60 con 0x7fa5980c6c30 2023-11-24T07:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59f0cc700 1 -- 172.21.15.32:0/4231617135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 msgr2=0x7fa5980cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/4231617135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5980cea00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5980c8c00 tx=0x7fa584009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59f0cc700 1 -- 172.21.15.32:0/4231617135 shutdown_connections 2023-11-24T07:59:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/4231617135 >> v2:172.21.15.123:3300/0 conn(0x7fa5980cef40 0x7fa5980cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/4231617135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5980cea00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.855+0000 7fa59f0cc700 1 -- 172.21.15.32:0/4231617135 >> 172.21.15.32:0/4231617135 conn(0x7fa5980b3780 msgr2=0x7fa5980b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.856+0000 7fa59f0cc700 1 -- 172.21.15.32:0/4231617135 shutdown_connections 2023-11-24T07:59:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.856+0000 7fa59f0cc700 1 -- 172.21.15.32:0/4231617135 wait complete. 2023-11-24T07:59:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.857+0000 7fa59f0cc700 1 Processor -- start 2023-11-24T07:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.857+0000 7fa59f0cc700 1 -- start start 2023-11-24T07:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5981540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5980cef40 0x7fa59809f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59815bf50 0x7fa5980b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59809de10 con 0x7fa5980c6c30 2023-11-24T07:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59809e230 con 0x7fa59815bf50 2023-11-24T07:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa598154620 con 0x7fa5980cef40 2023-11-24T07:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5981540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5981540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58390/0 (socket says 172.21.15.32:58390) 2023-11-24T07:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 -- 172.21.15.32:0/1353847889 learned_addr learned my addr 172.21.15.32:0/1353847889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa597fff700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5980cef40 0x7fa59809f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59d669700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59815bf50 0x7fa5980b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 -- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5980cef40 msgr2=0x7fa59809f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5980cef40 0x7fa59809f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 -- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59815bf50 msgr2=0x7fa5980b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59815bf50 0x7fa5980b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 -- 172.21.15.32:0/1353847889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009550 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59ce68700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5981540e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c450 tx=0x7fa58c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0052d0 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c005430 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59809f620 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.858+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0038c0 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.859+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59809d470 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.859+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa598068fb0 con 0x7fa5980c6c30 2023-11-24T07:59:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.860+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa58c003a20 con 0x7fa5980c6c30 2023-11-24T07:59:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.860+0000 7fa5957fa700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa57c06aed0 0x7fa57c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.860+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa58c0888f0 con 0x7fa5980c6c30 2023-11-24T07:59:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.861+0000 7fa597fff700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa57c06aed0 0x7fa57c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.861+0000 7fa597fff700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa57c06aed0 0x7fa57c06d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa584000c00 tx=0x7fa584008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:36.865+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa58c0539a0 con 0x7fa5980c6c30 2023-11-24T07:59:37.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.113+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa5980cdd00 con 0x7fa5980c6c30 2023-11-24T07:59:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.707+0000 7fa5957fa700 1 -- 172.21.15.32:0/1353847889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa5980cdd00 con 0x7fa5980c6c30 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa57c06aed0 msgr2=0x7fa57c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa57c06aed0 0x7fa57c06d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa584000c00 tx=0x7fa584008030 comp rx=0 tx=0).stop 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 msgr2=0x7fa5981540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5981540e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c450 tx=0x7fa58c00c760 comp rx=0 tx=0).stop 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 shutdown_connections 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa57c06aed0 0x7fa57c06d390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59815bf50 0x7fa5980b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5980c6c30 0x7fa5981540e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 --2- 172.21.15.32:0/1353847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5980cef40 0x7fa59809f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 >> 172.21.15.32:0/1353847889 conn(0x7fa5980b3780 msgr2=0x7fa59806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 shutdown_connections 2023-11-24T07:59:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.710+0000 7fa59f0cc700 1 -- 172.21.15.32:0/1353847889 wait complete. 2023-11-24T07:59:37.726 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:59:37.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:37.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.855+0000 7f1a102e5700 1 Processor -- start 2023-11-24T07:59:37.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.855+0000 7f1a102e5700 1 -- start start 2023-11-24T07:59:37.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a102e5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a0815bdd0 0x7f1a081604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a102e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a080d1c70 0x7f1a080d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a102e5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a080b22d0 con 0x7f1a080b3370 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a102e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a080b2f50 con 0x7f1a080d1c70 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a102e5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080d2640 con 0x7f1a0815bdd0 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a080d1c70 0x7f1a080d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a080d1c70 0x7f1a080d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58402/0 (socket says 172.21.15.32:58402) 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 -- 172.21.15.32:0/1666140347 learned_addr learned my addr 172.21.15.32:0/1666140347 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0d880700 1 --1- 172.21.15.32:0/1666140347 >> v1:172.21.15.32:6789/0 conn(0x7f1a080b3370 0x7f1a080b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60110/0 (socket says 172.21.15.32:60110) 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e081700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a0815bdd0 0x7f1a081604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 -- 172.21.15.32:0/1666140347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a0815bdd0 msgr2=0x7f1a081604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a0815bdd0 0x7f1a081604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 -- 172.21.15.32:0/1666140347 >> v1:172.21.15.32:6789/0 conn(0x7f1a080b3370 legacy=0x7f1a080b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e882700 1 -- 172.21.15.32:0/1666140347 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080b25b0 con 0x7f1a080d1c70 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.856+0000 7f1a0e081700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a0815bdd0 0x7f1a081604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a0e882700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a080d2100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009e50 tx=0x7f1a0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f02211badc261 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a0d07f700 1 -- 172.21.15.32:0/1666140347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000e070 con 0x7f1a080d1c70 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a0d07f700 1 -- 172.21.15.32:0/1666140347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0000b9c0 con 0x7f1a080d1c70 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a0d07f700 1 -- 172.21.15.32:0/1666140347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000bd70 con 0x7f1a080d1c70 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 -- 172.21.15.32:0/1666140347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 msgr2=0x7f1a080d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a080d2100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009e50 tx=0x7f1a0000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:37.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 -- 172.21.15.32:0/1666140347 shutdown_connections 2023-11-24T07:59:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a080d2100 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 --2- 172.21.15.32:0/1666140347 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a0815bdd0 0x7f1a081604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 -- 172.21.15.32:0/1666140347 >> 172.21.15.32:0/1666140347 conn(0x7f1a080731a0 msgr2=0x7f1a0809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 -- 172.21.15.32:0/1666140347 shutdown_connections 2023-11-24T07:59:37.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.857+0000 7f1a102e5700 1 -- 172.21.15.32:0/1666140347 wait complete. 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 Processor -- start 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 -- start start 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a080b3370 0x7f1a080c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a0815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a0815bdd0 0x7f1a0815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080b5990 con 0x7f1a080d1c70 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080c7e60 con 0x7f1a0815bdd0 2023-11-24T07:59:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a102e5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080c8140 con 0x7f1a080b3370 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a0d880700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a0815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a0d880700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a0815ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58406/0 (socket says 172.21.15.32:58406) 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.859+0000 7f1a0d880700 1 -- 172.21.15.32:0/4212233656 learned_addr learned my addr 172.21.15.32:0/4212233656 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0e081700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a080b3370 0x7f1a080c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0e882700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a0815bdd0 0x7f1a0815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0d880700 1 -- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a080b3370 msgr2=0x7f1a080c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0d880700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a080b3370 0x7f1a080c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0d880700 1 -- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a0815bdd0 msgr2=0x7f1a0815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0d880700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a0815bdd0 0x7f1a0815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0d880700 1 -- 172.21.15.32:0/4212233656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a00009b30 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a0d880700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a0815ec30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c520 tx=0x7f1a0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04005cb0 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a04005e10 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a080c8420 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040072d0 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.860+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a080c7b30 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.861+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a08067f60 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.862+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a04007430 con 0x7f1a080d1c70 2023-11-24T07:59:37.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.862+0000 7f19feffd700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f19ec06aed0 0x7f19ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.862+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a04088950 con 0x7f1a080d1c70 2023-11-24T07:59:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.862+0000 7f1a0e081700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f19ec06aed0 0x7f19ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.863+0000 7f1a0e081700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f19ec06aed0 0x7f19ec06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c900 tx=0x7f19f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:37.866+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a04053b40 con 0x7f1a080d1c70 2023-11-24T07:59:38.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.110+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a080b5ad0 con 0x7f1a080d1c70 2023-11-24T07:59:38.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.110+0000 7f19feffd700 1 -- 172.21.15.32:0/4212233656 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f1a080b5ad0 con 0x7f1a080d1c70 2023-11-24T07:59:38.112 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:38.112 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:34.106005+0000","modified":"2023-11-24T07:59:37.114690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/598023464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":598023464},{"type":"v1","addr":"172.21.15.32:6835","nonce":598023464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:59:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f19ec06aed0 msgr2=0x7f19ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f19ec06aed0 0x7f19ec06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c900 tx=0x7f19f8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 msgr2=0x7f1a0815ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a0815ec30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c520 tx=0x7f1a0400c8e0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 shutdown_connections 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f19ec06aed0 0x7f19ec06d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a0815bdd0 0x7f1a0815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a080b3370 0x7f1a080c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 --2- 172.21.15.32:0/4212233656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a080d1c70 0x7f1a0815ec30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 >> 172.21.15.32:0/4212233656 conn(0x7f1a080731a0 msgr2=0x7f1a080cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 shutdown_connections 2023-11-24T07:59:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.112+0000 7f1a102e5700 1 -- 172.21.15.32:0/4212233656 wait complete. 2023-11-24T07:59:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-11-24T07:59:38.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.252+0000 7f5302108700 1 Processor -- start 2023-11-24T07:59:38.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.252+0000 7f5302108700 1 -- start start 2023-11-24T07:59:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f5302108700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52fc15c140 0x7f52fc0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f5302108700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52fc0cf2b0 0x7f52fc0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f5302108700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0cc780 con 0x7f52fc15bd60 2023-11-24T07:59:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f5302108700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc0cc9b0 con 0x7f52fc0cf2b0 2023-11-24T07:59:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f5302108700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc0ccc90 con 0x7f52fc15c140 2023-11-24T07:59:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f52fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52fc0cf2b0 0x7f52fc0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f52fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52fc0cf2b0 0x7f52fc0c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58416/0 (socket says 172.21.15.32:58416) 2023-11-24T07:59:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.253+0000 7f52fbfff700 1 -- 172.21.15.32:0/2894405393 learned_addr learned my addr 172.21.15.32:0/2894405393 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52faffd700 1 --2- 172.21.15.32:0/2894405393 >> v2:172.21.15.123:3300/0 conn(0x7f52fc15c140 0x7f52fc0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fbfff700 1 -- 172.21.15.32:0/2894405393 >> v2:172.21.15.123:3300/0 conn(0x7f52fc15c140 msgr2=0x7f52fc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fbfff700 1 --2- 172.21.15.32:0/2894405393 >> v2:172.21.15.123:3300/0 conn(0x7f52fc15c140 0x7f52fc0cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fbfff700 1 -- 172.21.15.32:0/2894405393 >> v1:172.21.15.123:6789/0 conn(0x7f52fc15bd60 legacy=0x7f52fc0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fbfff700 1 -- 172.21.15.32:0/2894405393 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0ccf70 con 0x7f52fc0cf2b0 2023-11-24T07:59:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52faffd700 1 --2- 172.21.15.32:0/2894405393 >> v2:172.21.15.123:3300/0 conn(0x7f52fc15c140 0x7f52fc0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fbfff700 1 --2- 172.21.15.32:0/2894405393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc0c57c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0d2cd0 tx=0x7f52f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd08d3f36926e38 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fa7fc700 1 -- 172.21.15.32:0/2894405393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400e070 con 0x7f52fc0cf2b0 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fa7fc700 1 -- 172.21.15.32:0/2894405393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f400b9c0 con 0x7f52fc0cf2b0 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.254+0000 7f52fa7fc700 1 -- 172.21.15.32:0/2894405393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400bd70 con 0x7f52fc0cf2b0 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 -- 172.21.15.32:0/2894405393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc0cf2b0 msgr2=0x7f52fc0c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 --2- 172.21.15.32:0/2894405393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc0c57c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0d2cd0 tx=0x7f52f400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 -- 172.21.15.32:0/2894405393 shutdown_connections 2023-11-24T07:59:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 --2- 172.21.15.32:0/2894405393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc0c57c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 --2- 172.21.15.32:0/2894405393 >> v2:172.21.15.123:3300/0 conn(0x7f52fc15c140 0x7f52fc0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 -- 172.21.15.32:0/2894405393 >> 172.21.15.32:0/2894405393 conn(0x7f52fc073130 msgr2=0x7f52fc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 -- 172.21.15.32:0/2894405393 shutdown_connections 2023-11-24T07:59:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.255+0000 7f5302108700 1 -- 172.21.15.32:0/2894405393 wait complete. 2023-11-24T07:59:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.256+0000 7f5302108700 1 Processor -- start 2023-11-24T07:59:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.256+0000 7f5302108700 1 -- start start 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f5302108700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f5302108700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52fc15bd60 0x7f52fc09bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f5302108700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 0x7f52fc15d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f5302108700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc09fb20 con 0x7f52fc15c140 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f5302108700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc09ef10 con 0x7f52fc0cf2b0 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f5302108700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc09f1f0 con 0x7f52fc15bd60 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 0x7f52fc15d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 0x7f52fc15d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58422/0 (socket says 172.21.15.32:58422) 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 -- 172.21.15.32:0/540241429 learned_addr learned my addr 172.21.15.32:0/540241429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52faffd700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52fc15bd60 0x7f52fc09bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fb7fe700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 -- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52fc15bd60 msgr2=0x7f52fc09bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52fc15bd60 0x7f52fc09bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 -- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52fc0cf2b0 msgr2=0x7f52fc1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc1571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 -- 172.21.15.32:0/540241429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009b30 con 0x7f52fc15c140 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.257+0000 7f52fbfff700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 0x7f52fc15d5a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0730a0 tx=0x7f52f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.258+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40099b0 con 0x7f52fc15c140 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.258+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f40092e0 con 0x7f52fc15c140 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.258+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc09f4d0 con 0x7f52fc15c140 2023-11-24T07:59:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.258+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f401c640 con 0x7f52fc15c140 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.258+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc1576f0 con 0x7f52fc15c140 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.259+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc0ca190 con 0x7f52fc15c140 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.259+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52f401c7a0 con 0x7f52fc15c140 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.259+0000 7f52f3fff700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52dc06aed0 0x7f52dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.260+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52f408b2c0 con 0x7f52fc15c140 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.260+0000 7f52fb7fe700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52dc06aed0 0x7f52dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.260+0000 7f52fb7fe700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52dc06aed0 0x7f52dc06d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f52ec000f60 tx=0x7f52ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.264+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52f4056400 con 0x7f52fc15c140 2023-11-24T07:59:38.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.463+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52fc09fc60 con 0x7f52fc15c140 2023-11-24T07:59:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.464+0000 7f52f3fff700 1 -- 172.21.15.32:0/540241429 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f52fc09fc60 con 0x7f52fc15c140 2023-11-24T07:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:35.687434+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52dc06aed0 msgr2=0x7f52dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52dc06aed0 0x7f52dc06d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f52ec000f60 tx=0x7f52ec009040 comp rx=0 tx=0).stop 2023-11-24T07:59:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 msgr2=0x7f52fc15d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 0x7f52fc15d5a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0730a0 tx=0x7f52f4004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 shutdown_connections 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52dc06aed0 0x7f52dc06d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52fc15c140 0x7f52fc15d5a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52fc0cf2b0 0x7f52fc1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 --2- 172.21.15.32:0/540241429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52fc15bd60 0x7f52fc09bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 >> 172.21.15.32:0/540241429 conn(0x7f52fc073130 msgr2=0x7f52fc15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 shutdown_connections 2023-11-24T07:59:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.466+0000 7f5302108700 1 -- 172.21.15.32:0/540241429 wait complete. 2023-11-24T07:59:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:59:38.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:38.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.604+0000 7f466ed8d700 1 Processor -- start 2023-11-24T07:59:38.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.604+0000 7f466ed8d700 1 -- start start 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466ed8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f466815c140 0x7f46680d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466ed8d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4668074e90 con 0x7f466806ee50 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466ed8d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466815c700 con 0x7f4668073f90 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466ed8d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466815c9e0 con 0x7f466815c140 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f466815c140 0x7f46680d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f466815c140 0x7f46680d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58430/0 (socket says 172.21.15.32:58430) 2023-11-24T07:59:38.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 -- 172.21.15.32:0/748985494 learned_addr learned my addr 172.21.15.32:0/748985494 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 -- 172.21.15.32:0/748985494 >> v1:172.21.15.123:6789/0 conn(0x7f4668073f90 legacy=0x7f46680ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 -- 172.21.15.32:0/748985494 >> v1:172.21.15.32:6789/0 conn(0x7f466806ee50 legacy=0x7f4668074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 -- 172.21.15.32:0/748985494 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466815ccc0 con 0x7f466815c140 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.605+0000 7f466d32a700 1 --2- 172.21.15.32:0/748985494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466815c140 0x7f46680d02a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4660009e80 tx=0x7f466000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7a590ca0f1d1d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f46677fe700 1 -- 172.21.15.32:0/748985494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000e070 con 0x7f466815c140 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f46677fe700 1 -- 172.21.15.32:0/748985494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660003a20 con 0x7f466815c140 2023-11-24T07:59:38.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f46677fe700 1 -- 172.21.15.32:0/748985494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660003dd0 con 0x7f466815c140 2023-11-24T07:59:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 -- 172.21.15.32:0/748985494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466815c140 msgr2=0x7f46680d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 --2- 172.21.15.32:0/748985494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466815c140 0x7f46680d02a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4660009e80 tx=0x7f466000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 -- 172.21.15.32:0/748985494 shutdown_connections 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 --2- 172.21.15.32:0/748985494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466815c140 0x7f46680d02a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 -- 172.21.15.32:0/748985494 >> 172.21.15.32:0/748985494 conn(0x7f46680cd570 msgr2=0x7f46680cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 -- 172.21.15.32:0/748985494 shutdown_connections 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.606+0000 7f466ed8d700 1 -- 172.21.15.32:0/748985494 wait complete. 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.607+0000 7f466ed8d700 1 Processor -- start 2023-11-24T07:59:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.607+0000 7f466ed8d700 1 -- start start 2023-11-24T07:59:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466ed8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 0x7f46681589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466ed8d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4668073f90 0x7f466814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466ed8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f466815c140 0x7f466814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466ed8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681654f0 con 0x7f466806ee50 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466ed8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668151790 con 0x7f466815c140 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466ed8d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46681511d0 con 0x7f4668073f90 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466cb29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 0x7f46681589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466cb29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 0x7f46681589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58432/0 (socket says 172.21.15.32:58432) 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.608+0000 7f466cb29700 1 -- 172.21.15.32:0/4435589 learned_addr learned my addr 172.21.15.32:0/4435589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466cb29700 1 -- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4668073f90 msgr2=0x7f466814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466cb29700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4668073f90 0x7f466814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466cb29700 1 -- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f466815c140 msgr2=0x7f466814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466cb29700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f466815c140 0x7f466814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466cb29700 1 -- 172.21.15.32:0/4435589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660009b30 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466cb29700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 0x7f46681589c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c4d0 tx=0x7f465c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011790 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0118f0 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46681514b0 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011ca0 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.609+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46680b3be0 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.610+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668067f60 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.610+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f465c01c460 con 0x7f466806ee50 2023-11-24T07:59:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.611+0000 7f46657fa700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f464c06aed0 0x7f464c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.611+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f465c089420 con 0x7f466806ee50 2023-11-24T07:59:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.611+0000 7f4667fff700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f464c06aed0 0x7f464c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.612+0000 7f4667fff700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f464c06aed0 0x7f464c06d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4654000f60 tx=0x7f4654007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.616+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f465c054560 con 0x7f466806ee50 2023-11-24T07:59:38.671 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-11-24T07:59:38.670+0000 7f5b51656700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:59:38.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.815+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46680b3da0 con 0x7f464c06aed0 2023-11-24T07:59:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.825+0000 7f46657fa700 1 -- 172.21.15.32:0/4435589 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288654 (secure 0 0 0) 0x7f46680b3da0 con 0x7f464c06aed0 2023-11-24T07:59:38.830 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f464c06aed0 msgr2=0x7f464c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f464c06aed0 0x7f464c06d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4654000f60 tx=0x7f4654007a60 comp rx=0 tx=0).stop 2023-11-24T07:59:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 msgr2=0x7f46681589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 0x7f46681589c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c4d0 tx=0x7f465c00c890 comp rx=0 tx=0).stop 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 shutdown_connections 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f464c06aed0 0x7f464c06d390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f466815c140 0x7f466814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466806ee50 0x7f46681589c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 --2- 172.21.15.32:0/4435589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4668073f90 0x7f466814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 >> 172.21.15.32:0/4435589 conn(0x7f46680cd570 msgr2=0x7f466814eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 shutdown_connections 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:38.828+0000 7f466ed8d700 1 -- 172.21.15.32:0/4435589 wait complete. 2023-11-24T07:59:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T07:59:38.842 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-11-24T07:59:38.544987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220184,"kb_used_data":4004,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754536,"statfs":{"total":773094113280,"available":772868644864,"internally_reserved":0,"allocated":4100096,"data_stored":2276840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.866103"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702699+0000","last_change":"2023-11-24T07:59:34.696902+0000","last_active":"2023-11-24T07:59:35.702699+0000","last_peered":"2023-11-24T07:59:35.702699+0000","last_clean":"2023-11-24T07:59:35.702699+0000","last_became_active":"2023-11-24T07:59:34.696588+0000","last_became_peered":"2023-11-24T07:59:34.696588+0000","last_unstale":"2023-11-24T07:59:35.702699+0000","last_undegraded":"2023-11-24T07:59:35.702699+0000","last_fullsized":"2023-11-24T07:59:35.702699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:58.348867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698786+0000","last_change":"2023-11-24T07:59:34.708947+0000","last_active":"2023-11-24T07:59:35.698786+0000","last_peered":"2023-11-24T07:59:35.698786+0000","last_clean":"2023-11-24T07:59:35.698786+0000","last_became_active":"2023-11-24T07:59:34.708625+0000","last_became_peered":"2023-11-24T07:59:34.708625+0000","last_unstale":"2023-11-24T07:59:35.698786+0000","last_undegraded":"2023-11-24T07:59:35.698786+0000","last_fullsized":"2023-11-24T07:59:35.698786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:25.675685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701945+0000","last_change":"2023-11-24T07:59:34.702426+0000","last_active":"2023-11-24T07:59:35.701945+0000","last_peered":"2023-11-24T07:59:35.701945+0000","last_clean":"2023-11-24T07:59:35.701945+0000","last_became_active":"2023-11-24T07:59:34.702161+0000","last_became_peered":"2023-11-24T07:59:34.702161+0000","last_unstale":"2023-11-24T07:59:35.701945+0000","last_undegraded":"2023-11-24T07:59:35.701945+0000","last_fullsized":"2023-11-24T07:59:35.701945+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317054+0000","last_change":"2023-11-24T07:59:34.707996+0000","last_active":"2023-11-24T07:59:36.317054+0000","last_peered":"2023-11-24T07:59:36.317054+0000","last_clean":"2023-11-24T07:59:36.317054+0000","last_became_active":"2023-11-24T07:59:34.707720+0000","last_became_peered":"2023-11-24T07:59:34.707720+0000","last_unstale":"2023-11-24T07:59:36.317054+0000","last_undegraded":"2023-11-24T07:59:36.317054+0000","last_fullsized":"2023-11-24T07:59:36.317054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:09.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316356+0000","last_change":"2023-11-24T07:59:34.709389+0000","last_active":"2023-11-24T07:59:36.316356+0000","last_peered":"2023-11-24T07:59:36.316356+0000","last_clean":"2023-11-24T07:59:36.316356+0000","last_became_active":"2023-11-24T07:59:34.708883+0000","last_became_peered":"2023-11-24T07:59:34.708883+0000","last_unstale":"2023-11-24T07:59:36.316356+0000","last_undegraded":"2023-11-24T07:59:36.316356+0000","last_fullsized":"2023-11-24T07:59:36.316356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:40.366654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698537+0000","last_change":"2023-11-24T07:59:34.695237+0000","last_active":"2023-11-24T07:59:35.698537+0000","last_peered":"2023-11-24T07:59:35.698537+0000","last_clean":"2023-11-24T07:59:35.698537+0000","last_became_active":"2023-11-24T07:59:34.694926+0000","last_became_peered":"2023-11-24T07:59:34.694926+0000","last_unstale":"2023-11-24T07:59:35.698537+0000","last_undegraded":"2023-11-24T07:59:35.698537+0000","last_fullsized":"2023-11-24T07:59:35.698537+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:48.172772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316785+0000","last_change":"2023-11-24T07:59:34.706621+0000","last_active":"2023-11-24T07:59:36.316785+0000","last_peered":"2023-11-24T07:59:36.316785+0000","last_clean":"2023-11-24T07:59:36.316785+0000","last_became_active":"2023-11-24T07:59:34.706364+0000","last_became_peered":"2023-11-24T07:59:34.706364+0000","last_unstale":"2023-11-24T07:59:36.316785+0000","last_undegraded":"2023-11-24T07:59:36.316785+0000","last_fullsized":"2023-11-24T07:59:36.316785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:10.747185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316747+0000","last_change":"2023-11-24T07:59:34.706447+0000","last_active":"2023-11-24T07:59:36.316747+0000","last_peered":"2023-11-24T07:59:36.316747+0000","last_clean":"2023-11-24T07:59:36.316747+0000","last_became_active":"2023-11-24T07:59:34.705935+0000","last_became_peered":"2023-11-24T07:59:34.705935+0000","last_unstale":"2023-11-24T07:59:36.316747+0000","last_undegraded":"2023-11-24T07:59:36.316747+0000","last_fullsized":"2023-11-24T07:59:36.316747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:01.322460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317328+0000","last_change":"2023-11-24T07:59:34.716245+0000","last_active":"2023-11-24T07:59:36.317328+0000","last_peered":"2023-11-24T07:59:36.317328+0000","last_clean":"2023-11-24T07:59:36.317328+0000","last_became_active":"2023-11-24T07:59:34.715942+0000","last_became_peered":"2023-11-24T07:59:34.715942+0000","last_unstale":"2023-11-24T07:59:36.317328+0000","last_undegraded":"2023-11-24T07:59:36.317328+0000","last_fullsized":"2023-11-24T07:59:36.317328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:54.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317198+0000","last_change":"2023-11-24T07:59:34.709403+0000","last_active":"2023-11-24T07:59:36.317198+0000","last_peered":"2023-11-24T07:59:36.317198+0000","last_clean":"2023-11-24T07:59:36.317198+0000","last_became_active":"2023-11-24T07:59:34.709097+0000","last_became_peered":"2023-11-24T07:59:34.709097+0000","last_unstale":"2023-11-24T07:59:36.317198+0000","last_undegraded":"2023-11-24T07:59:36.317198+0000","last_fullsized":"2023-11-24T07:59:36.317198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:41.599941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317009+0000","last_change":"2023-11-24T07:59:34.702388+0000","last_active":"2023-11-24T07:59:36.317009+0000","last_peered":"2023-11-24T07:59:36.317009+0000","last_clean":"2023-11-24T07:59:36.317009+0000","last_became_active":"2023-11-24T07:59:34.702105+0000","last_became_peered":"2023-11-24T07:59:34.702105+0000","last_unstale":"2023-11-24T07:59:36.317009+0000","last_undegraded":"2023-11-24T07:59:36.317009+0000","last_fullsized":"2023-11-24T07:59:36.317009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:48.172772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702541+0000","last_change":"2023-11-24T07:59:34.700871+0000","last_active":"2023-11-24T07:59:35.702541+0000","last_peered":"2023-11-24T07:59:35.702541+0000","last_clean":"2023-11-24T07:59:35.702541+0000","last_became_active":"2023-11-24T07:59:34.700241+0000","last_became_peered":"2023-11-24T07:59:34.700241+0000","last_unstale":"2023-11-24T07:59:35.702541+0000","last_undegraded":"2023-11-24T07:59:35.702541+0000","last_fullsized":"2023-11-24T07:59:35.702541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:18.983207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316621+0000","last_change":"2023-11-24T07:59:34.703008+0000","last_active":"2023-11-24T07:59:36.316621+0000","last_peered":"2023-11-24T07:59:36.316621+0000","last_clean":"2023-11-24T07:59:36.316621+0000","last_became_active":"2023-11-24T07:59:34.702674+0000","last_became_peered":"2023-11-24T07:59:34.702674+0000","last_unstale":"2023-11-24T07:59:36.316621+0000","last_undegraded":"2023-11-24T07:59:36.316621+0000","last_fullsized":"2023-11-24T07:59:36.316621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:13.819295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698568+0000","last_change":"2023-11-24T07:59:34.710511+0000","last_active":"2023-11-24T07:59:35.698568+0000","last_peered":"2023-11-24T07:59:35.698568+0000","last_clean":"2023-11-24T07:59:35.698568+0000","last_became_active":"2023-11-24T07:59:34.709859+0000","last_became_peered":"2023-11-24T07:59:34.709859+0000","last_unstale":"2023-11-24T07:59:35.698568+0000","last_undegraded":"2023-11-24T07:59:35.698568+0000","last_fullsized":"2023-11-24T07:59:35.698568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:18.811326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702407+0000","last_change":"2023-11-24T07:59:34.706291+0000","last_active":"2023-11-24T07:59:35.702407+0000","last_peered":"2023-11-24T07:59:35.702407+0000","last_clean":"2023-11-24T07:59:35.702407+0000","last_became_active":"2023-11-24T07:59:34.705978+0000","last_became_peered":"2023-11-24T07:59:34.705978+0000","last_unstale":"2023-11-24T07:59:35.702407+0000","last_undegraded":"2023-11-24T07:59:35.702407+0000","last_fullsized":"2023-11-24T07:59:35.702407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:31.838023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316591+0000","last_change":"2023-11-24T07:59:34.710684+0000","last_active":"2023-11-24T07:59:36.316591+0000","last_peered":"2023-11-24T07:59:36.316591+0000","last_clean":"2023-11-24T07:59:36.316591+0000","last_became_active":"2023-11-24T07:59:34.710435+0000","last_became_peered":"2023-11-24T07:59:34.710435+0000","last_unstale":"2023-11-24T07:59:36.316591+0000","last_undegraded":"2023-11-24T07:59:36.316591+0000","last_fullsized":"2023-11-24T07:59:36.316591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316984+0000","last_change":"2023-11-24T07:59:34.707171+0000","last_active":"2023-11-24T07:59:36.316984+0000","last_peered":"2023-11-24T07:59:36.316984+0000","last_clean":"2023-11-24T07:59:36.316984+0000","last_became_active":"2023-11-24T07:59:34.706918+0000","last_became_peered":"2023-11-24T07:59:34.706918+0000","last_unstale":"2023-11-24T07:59:36.316984+0000","last_undegraded":"2023-11-24T07:59:36.316984+0000","last_fullsized":"2023-11-24T07:59:36.316984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:18.983207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698781+0000","last_change":"2023-11-24T07:59:34.706692+0000","last_active":"2023-11-24T07:59:35.698781+0000","last_peered":"2023-11-24T07:59:35.698781+0000","last_clean":"2023-11-24T07:59:35.698781+0000","last_became_active":"2023-11-24T07:59:34.706277+0000","last_became_peered":"2023-11-24T07:59:34.706277+0000","last_unstale":"2023-11-24T07:59:35.698781+0000","last_undegraded":"2023-11-24T07:59:35.698781+0000","last_fullsized":"2023-11-24T07:59:35.698781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:42.122406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698488+0000","last_change":"2023-11-24T07:59:34.693674+0000","last_active":"2023-11-24T07:59:35.698488+0000","last_peered":"2023-11-24T07:59:35.698488+0000","last_clean":"2023-11-24T07:59:35.698488+0000","last_became_active":"2023-11-24T07:59:34.693387+0000","last_became_peered":"2023-11-24T07:59:34.693387+0000","last_unstale":"2023-11-24T07:59:35.698488+0000","last_undegraded":"2023-11-24T07:59:35.698488+0000","last_fullsized":"2023-11-24T07:59:35.698488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:18.983207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316679+0000","last_change":"2023-11-24T07:59:34.707725+0000","last_active":"2023-11-24T07:59:36.316679+0000","last_peered":"2023-11-24T07:59:36.316679+0000","last_clean":"2023-11-24T07:59:36.316679+0000","last_became_active":"2023-11-24T07:59:34.707395+0000","last_became_peered":"2023-11-24T07:59:34.707395+0000","last_unstale":"2023-11-24T07:59:36.316679+0000","last_undegraded":"2023-11-24T07:59:36.316679+0000","last_fullsized":"2023-11-24T07:59:36.316679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:06.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317074+0000","last_change":"2023-11-24T07:59:34.712358+0000","last_active":"2023-11-24T07:59:36.317074+0000","last_peered":"2023-11-24T07:59:36.317074+0000","last_clean":"2023-11-24T07:59:36.317074+0000","last_became_active":"2023-11-24T07:59:34.712098+0000","last_became_peered":"2023-11-24T07:59:34.712098+0000","last_unstale":"2023-11-24T07:59:36.317074+0000","last_undegraded":"2023-11-24T07:59:36.317074+0000","last_fullsized":"2023-11-24T07:59:36.317074+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:09.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317331+0000","last_change":"2023-11-24T07:59:34.710729+0000","last_active":"2023-11-24T07:59:36.317331+0000","last_peered":"2023-11-24T07:59:36.317331+0000","last_clean":"2023-11-24T07:59:36.317331+0000","last_became_active":"2023-11-24T07:59:34.710425+0000","last_became_peered":"2023-11-24T07:59:34.710425+0000","last_unstale":"2023-11-24T07:59:36.317331+0000","last_undegraded":"2023-11-24T07:59:36.317331+0000","last_fullsized":"2023-11-24T07:59:36.317331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:21.250798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.315933+0000","last_change":"2023-11-24T07:59:34.705414+0000","last_active":"2023-11-24T07:59:36.315933+0000","last_peered":"2023-11-24T07:59:36.315933+0000","last_clean":"2023-11-24T07:59:36.315933+0000","last_became_active":"2023-11-24T07:59:34.704888+0000","last_became_peered":"2023-11-24T07:59:34.704888+0000","last_unstale":"2023-11-24T07:59:36.315933+0000","last_undegraded":"2023-11-24T07:59:36.315933+0000","last_fullsized":"2023-11-24T07:59:36.315933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:57.668606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701794+0000","last_change":"2023-11-24T07:59:34.695742+0000","last_active":"2023-11-24T07:59:35.701794+0000","last_peered":"2023-11-24T07:59:35.701794+0000","last_clean":"2023-11-24T07:59:35.701794+0000","last_became_active":"2023-11-24T07:59:34.695436+0000","last_became_peered":"2023-11-24T07:59:34.695436+0000","last_unstale":"2023-11-24T07:59:35.701794+0000","last_undegraded":"2023-11-24T07:59:35.701794+0000","last_fullsized":"2023-11-24T07:59:35.701794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317356+0000","last_change":"2023-11-24T07:59:34.715895+0000","last_active":"2023-11-24T07:59:36.317356+0000","last_peered":"2023-11-24T07:59:36.317356+0000","last_clean":"2023-11-24T07:59:36.317356+0000","last_became_active":"2023-11-24T07:59:34.712482+0000","last_became_peered":"2023-11-24T07:59:34.712482+0000","last_unstale":"2023-11-24T07:59:36.317356+0000","last_undegraded":"2023-11-24T07:59:36.317356+0000","last_fullsized":"2023-11-24T07:59:36.317356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698871+0000","last_change":"2023-11-24T07:59:34.708051+0000","last_active":"2023-11-24T07:59:35.698871+0000","last_peered":"2023-11-24T07:59:35.698871+0000","last_clean":"2023-11-24T07:59:35.698871+0000","last_became_active":"2023-11-24T07:59:34.707760+0000","last_became_peered":"2023-11-24T07:59:34.707760+0000","last_unstale":"2023-11-24T07:59:35.698871+0000","last_undegraded":"2023-11-24T07:59:35.698871+0000","last_fullsized":"2023-11-24T07:59:35.698871+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:19.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317408+0000","last_change":"2023-11-24T07:59:34.717264+0000","last_active":"2023-11-24T07:59:36.317408+0000","last_peered":"2023-11-24T07:59:36.317408+0000","last_clean":"2023-11-24T07:59:36.317408+0000","last_became_active":"2023-11-24T07:59:34.716990+0000","last_became_peered":"2023-11-24T07:59:34.716990+0000","last_unstale":"2023-11-24T07:59:36.317408+0000","last_undegraded":"2023-11-24T07:59:36.317408+0000","last_fullsized":"2023-11-24T07:59:36.317408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702616+0000","last_change":"2023-11-24T07:59:34.701078+0000","last_active":"2023-11-24T07:59:35.702616+0000","last_peered":"2023-11-24T07:59:35.702616+0000","last_clean":"2023-11-24T07:59:35.702616+0000","last_became_active":"2023-11-24T07:59:34.700628+0000","last_became_peered":"2023-11-24T07:59:34.700628+0000","last_unstale":"2023-11-24T07:59:35.702616+0000","last_undegraded":"2023-11-24T07:59:35.702616+0000","last_fullsized":"2023-11-24T07:59:35.702616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:48.172772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317207+0000","last_change":"2023-11-24T07:59:34.707960+0000","last_active":"2023-11-24T07:59:36.317207+0000","last_peered":"2023-11-24T07:59:36.317207+0000","last_clean":"2023-11-24T07:59:36.317207+0000","last_became_active":"2023-11-24T07:59:34.707503+0000","last_became_peered":"2023-11-24T07:59:34.707503+0000","last_unstale":"2023-11-24T07:59:36.317207+0000","last_undegraded":"2023-11-24T07:59:36.317207+0000","last_fullsized":"2023-11-24T07:59:36.317207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:41.599941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316638+0000","last_change":"2023-11-24T07:59:34.706288+0000","last_active":"2023-11-24T07:59:36.316638+0000","last_peered":"2023-11-24T07:59:36.316638+0000","last_clean":"2023-11-24T07:59:36.316638+0000","last_became_active":"2023-11-24T07:59:34.706002+0000","last_became_peered":"2023-11-24T07:59:34.706002+0000","last_unstale":"2023-11-24T07:59:36.316638+0000","last_undegraded":"2023-11-24T07:59:36.316638+0000","last_fullsized":"2023-11-24T07:59:36.316638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702635+0000","last_change":"2023-11-24T07:59:34.705334+0000","last_active":"2023-11-24T07:59:35.702635+0000","last_peered":"2023-11-24T07:59:35.702635+0000","last_clean":"2023-11-24T07:59:35.702635+0000","last_became_active":"2023-11-24T07:59:34.704910+0000","last_became_peered":"2023-11-24T07:59:34.704910+0000","last_unstale":"2023-11-24T07:59:35.702635+0000","last_undegraded":"2023-11-24T07:59:35.702635+0000","last_fullsized":"2023-11-24T07:59:35.702635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:57.668606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316436+0000","last_change":"2023-11-24T07:59:34.702167+0000","last_active":"2023-11-24T07:59:36.316436+0000","last_peered":"2023-11-24T07:59:36.316436+0000","last_clean":"2023-11-24T07:59:36.316436+0000","last_became_active":"2023-11-24T07:59:34.701876+0000","last_became_peered":"2023-11-24T07:59:34.701876+0000","last_unstale":"2023-11-24T07:59:36.316436+0000","last_undegraded":"2023-11-24T07:59:36.316436+0000","last_fullsized":"2023-11-24T07:59:36.316436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:54.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316542+0000","last_change":"2023-11-24T07:59:34.705609+0000","last_active":"2023-11-24T07:59:36.316542+0000","last_peered":"2023-11-24T07:59:36.316542+0000","last_clean":"2023-11-24T07:59:36.316542+0000","last_became_active":"2023-11-24T07:59:34.704833+0000","last_became_peered":"2023-11-24T07:59:34.704833+0000","last_unstale":"2023-11-24T07:59:36.316542+0000","last_undegraded":"2023-11-24T07:59:36.316542+0000","last_fullsized":"2023-11-24T07:59:36.316542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:04.322801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702452+0000","last_change":"2023-11-24T07:59:33.696080+0000","last_active":"2023-11-24T07:59:35.702452+0000","last_peered":"2023-11-24T07:59:35.702452+0000","last_clean":"2023-11-24T07:59:35.702452+0000","last_became_active":"2023-11-24T07:59:33.695704+0000","last_became_peered":"2023-11-24T07:59:33.695704+0000","last_unstale":"2023-11-24T07:59:35.702452+0000","last_undegraded":"2023-11-24T07:59:35.702452+0000","last_fullsized":"2023-11-24T07:59:35.702452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:18.330853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702346+0000","last_change":"2023-11-24T07:59:34.707387+0000","last_active":"2023-11-24T07:59:35.702346+0000","last_peered":"2023-11-24T07:59:35.702346+0000","last_clean":"2023-11-24T07:59:35.702346+0000","last_became_active":"2023-11-24T07:59:34.705529+0000","last_became_peered":"2023-11-24T07:59:34.705529+0000","last_unstale":"2023-11-24T07:59:35.702346+0000","last_undegraded":"2023-11-24T07:59:35.702346+0000","last_fullsized":"2023-11-24T07:59:35.702346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:57.274625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316595+0000","last_change":"2023-11-24T07:59:33.702646+0000","last_active":"2023-11-24T07:59:36.316595+0000","last_peered":"2023-11-24T07:59:36.316595+0000","last_clean":"2023-11-24T07:59:36.316595+0000","last_became_active":"2023-11-24T07:59:33.702376+0000","last_became_peered":"2023-11-24T07:59:33.702376+0000","last_unstale":"2023-11-24T07:59:36.316595+0000","last_undegraded":"2023-11-24T07:59:36.316595+0000","last_fullsized":"2023-11-24T07:59:36.316595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:18.330853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317335+0000","last_change":"2023-11-24T07:59:34.710719+0000","last_active":"2023-11-24T07:59:36.317335+0000","last_peered":"2023-11-24T07:59:36.317335+0000","last_clean":"2023-11-24T07:59:36.317335+0000","last_became_active":"2023-11-24T07:59:34.710128+0000","last_became_peered":"2023-11-24T07:59:34.710128+0000","last_unstale":"2023-11-24T07:59:36.317335+0000","last_undegraded":"2023-11-24T07:59:36.317335+0000","last_fullsized":"2023-11-24T07:59:36.317335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698521+0000","last_change":"2023-11-24T07:59:33.699942+0000","last_active":"2023-11-24T07:59:35.698521+0000","last_peered":"2023-11-24T07:59:35.698521+0000","last_clean":"2023-11-24T07:59:35.698521+0000","last_became_active":"2023-11-24T07:59:33.699618+0000","last_became_peered":"2023-11-24T07:59:33.699618+0000","last_unstale":"2023-11-24T07:59:35.698521+0000","last_undegraded":"2023-11-24T07:59:35.698521+0000","last_fullsized":"2023-11-24T07:59:35.698521+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:56.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317405+0000","last_change":"2023-11-24T07:59:34.706643+0000","last_active":"2023-11-24T07:59:36.317405+0000","last_peered":"2023-11-24T07:59:36.317405+0000","last_clean":"2023-11-24T07:59:36.317405+0000","last_became_active":"2023-11-24T07:59:34.705694+0000","last_became_peered":"2023-11-24T07:59:34.705694+0000","last_unstale":"2023-11-24T07:59:36.317405+0000","last_undegraded":"2023-11-24T07:59:36.317405+0000","last_fullsized":"2023-11-24T07:59:36.317405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.708979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316551+0000","last_change":"2023-11-24T07:59:33.700687+0000","last_active":"2023-11-24T07:59:36.316551+0000","last_peered":"2023-11-24T07:59:36.316551+0000","last_clean":"2023-11-24T07:59:36.316551+0000","last_became_active":"2023-11-24T07:59:33.700397+0000","last_became_peered":"2023-11-24T07:59:33.700397+0000","last_unstale":"2023-11-24T07:59:36.316551+0000","last_undegraded":"2023-11-24T07:59:36.316551+0000","last_fullsized":"2023-11-24T07:59:36.316551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:40.733686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316824+0000","last_change":"2023-11-24T07:59:34.702888+0000","last_active":"2023-11-24T07:59:36.316824+0000","last_peered":"2023-11-24T07:59:36.316824+0000","last_clean":"2023-11-24T07:59:36.316824+0000","last_became_active":"2023-11-24T07:59:34.702621+0000","last_became_peered":"2023-11-24T07:59:34.702621+0000","last_unstale":"2023-11-24T07:59:36.316824+0000","last_undegraded":"2023-11-24T07:59:36.316824+0000","last_fullsized":"2023-11-24T07:59:36.316824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:59.781689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317198+0000","last_change":"2023-11-24T07:59:33.696095+0000","last_active":"2023-11-24T07:59:36.317198+0000","last_peered":"2023-11-24T07:59:36.317198+0000","last_clean":"2023-11-24T07:59:36.317198+0000","last_became_active":"2023-11-24T07:59:33.695797+0000","last_became_peered":"2023-11-24T07:59:33.695797+0000","last_unstale":"2023-11-24T07:59:36.317198+0000","last_undegraded":"2023-11-24T07:59:36.317198+0000","last_fullsized":"2023-11-24T07:59:36.317198+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:37.050060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702111+0000","last_change":"2023-11-24T07:59:34.700656+0000","last_active":"2023-11-24T07:59:35.702111+0000","last_peered":"2023-11-24T07:59:35.702111+0000","last_clean":"2023-11-24T07:59:35.702111+0000","last_became_active":"2023-11-24T07:59:34.700355+0000","last_became_peered":"2023-11-24T07:59:34.700355+0000","last_unstale":"2023-11-24T07:59:35.702111+0000","last_undegraded":"2023-11-24T07:59:35.702111+0000","last_fullsized":"2023-11-24T07:59:35.702111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:01.322460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317147+0000","last_change":"2023-11-24T07:59:33.696178+0000","last_active":"2023-11-24T07:59:36.317147+0000","last_peered":"2023-11-24T07:59:36.317147+0000","last_clean":"2023-11-24T07:59:36.317147+0000","last_became_active":"2023-11-24T07:59:33.695039+0000","last_became_peered":"2023-11-24T07:59:33.695039+0000","last_unstale":"2023-11-24T07:59:36.317147+0000","last_undegraded":"2023-11-24T07:59:36.317147+0000","last_fullsized":"2023-11-24T07:59:36.317147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:22.490220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316593+0000","last_change":"2023-11-24T07:59:34.698776+0000","last_active":"2023-11-24T07:59:36.316593+0000","last_peered":"2023-11-24T07:59:36.316593+0000","last_clean":"2023-11-24T07:59:36.316593+0000","last_became_active":"2023-11-24T07:59:34.698504+0000","last_became_peered":"2023-11-24T07:59:34.698504+0000","last_unstale":"2023-11-24T07:59:36.316593+0000","last_undegraded":"2023-11-24T07:59:36.316593+0000","last_fullsized":"2023-11-24T07:59:36.316593+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:04.322801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698142+0000","last_change":"2023-11-24T07:59:33.689147+0000","last_active":"2023-11-24T07:59:35.698142+0000","last_peered":"2023-11-24T07:59:35.698142+0000","last_clean":"2023-11-24T07:59:35.698142+0000","last_became_active":"2023-11-24T07:59:33.688823+0000","last_became_peered":"2023-11-24T07:59:33.688823+0000","last_unstale":"2023-11-24T07:59:35.698142+0000","last_undegraded":"2023-11-24T07:59:35.698142+0000","last_fullsized":"2023-11-24T07:59:35.698142+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:56.663996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316861+0000","last_change":"2023-11-24T07:59:34.694778+0000","last_active":"2023-11-24T07:59:36.316861+0000","last_peered":"2023-11-24T07:59:36.316861+0000","last_clean":"2023-11-24T07:59:36.316861+0000","last_became_active":"2023-11-24T07:59:34.694462+0000","last_became_peered":"2023-11-24T07:59:34.694462+0000","last_unstale":"2023-11-24T07:59:36.316861+0000","last_undegraded":"2023-11-24T07:59:36.316861+0000","last_fullsized":"2023-11-24T07:59:36.316861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:43.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316442+0000","last_change":"2023-11-24T07:59:33.696084+0000","last_active":"2023-11-24T07:59:36.316442+0000","last_peered":"2023-11-24T07:59:36.316442+0000","last_clean":"2023-11-24T07:59:36.316442+0000","last_became_active":"2023-11-24T07:59:33.694868+0000","last_became_peered":"2023-11-24T07:59:33.694868+0000","last_unstale":"2023-11-24T07:59:36.316442+0000","last_undegraded":"2023-11-24T07:59:36.316442+0000","last_fullsized":"2023-11-24T07:59:36.316442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:03.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702523+0000","last_change":"2023-11-24T07:59:34.708850+0000","last_active":"2023-11-24T07:59:35.702523+0000","last_peered":"2023-11-24T07:59:35.702523+0000","last_clean":"2023-11-24T07:59:35.702523+0000","last_became_active":"2023-11-24T07:59:34.708597+0000","last_became_peered":"2023-11-24T07:59:34.708597+0000","last_unstale":"2023-11-24T07:59:35.702523+0000","last_undegraded":"2023-11-24T07:59:35.702523+0000","last_fullsized":"2023-11-24T07:59:35.702523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:18.811326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698742+0000","last_change":"2023-11-24T07:59:33.702602+0000","last_active":"2023-11-24T07:59:35.698742+0000","last_peered":"2023-11-24T07:59:35.698742+0000","last_clean":"2023-11-24T07:59:35.698742+0000","last_became_active":"2023-11-24T07:59:33.702316+0000","last_became_peered":"2023-11-24T07:59:33.702316+0000","last_unstale":"2023-11-24T07:59:35.698742+0000","last_undegraded":"2023-11-24T07:59:35.698742+0000","last_fullsized":"2023-11-24T07:59:35.698742+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:12:54.321416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698839+0000","last_change":"2023-11-24T07:59:34.717202+0000","last_active":"2023-11-24T07:59:35.698839+0000","last_peered":"2023-11-24T07:59:35.698839+0000","last_clean":"2023-11-24T07:59:35.698839+0000","last_became_active":"2023-11-24T07:59:34.716863+0000","last_became_peered":"2023-11-24T07:59:34.716863+0000","last_unstale":"2023-11-24T07:59:35.698839+0000","last_undegraded":"2023-11-24T07:59:35.698839+0000","last_fullsized":"2023-11-24T07:59:35.698839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:54.979019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316638+0000","last_change":"2023-11-24T07:59:33.694731+0000","last_active":"2023-11-24T07:59:36.316638+0000","last_peered":"2023-11-24T07:59:36.316638+0000","last_clean":"2023-11-24T07:59:36.316638+0000","last_became_active":"2023-11-24T07:59:33.694386+0000","last_became_peered":"2023-11-24T07:59:33.694386+0000","last_unstale":"2023-11-24T07:59:36.316638+0000","last_undegraded":"2023-11-24T07:59:36.316638+0000","last_fullsized":"2023-11-24T07:59:36.316638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:12:54.321416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698530+0000","last_change":"2023-11-24T07:59:34.707625+0000","last_active":"2023-11-24T07:59:35.698530+0000","last_peered":"2023-11-24T07:59:35.698530+0000","last_clean":"2023-11-24T07:59:35.698530+0000","last_became_active":"2023-11-24T07:59:34.707337+0000","last_became_peered":"2023-11-24T07:59:34.707337+0000","last_unstale":"2023-11-24T07:59:35.698530+0000","last_undegraded":"2023-11-24T07:59:35.698530+0000","last_fullsized":"2023-11-24T07:59:35.698530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:59.781689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702076+0000","last_change":"2023-11-24T07:59:33.702355+0000","last_active":"2023-11-24T07:59:35.702076+0000","last_peered":"2023-11-24T07:59:35.702076+0000","last_clean":"2023-11-24T07:59:35.702076+0000","last_became_active":"2023-11-24T07:59:33.702011+0000","last_became_peered":"2023-11-24T07:59:33.702011+0000","last_unstale":"2023-11-24T07:59:35.702076+0000","last_undegraded":"2023-11-24T07:59:35.702076+0000","last_fullsized":"2023-11-24T07:59:35.702076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:53.702419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701989+0000","last_change":"2023-11-24T07:59:34.702344+0000","last_active":"2023-11-24T07:59:35.701989+0000","last_peered":"2023-11-24T07:59:35.701989+0000","last_clean":"2023-11-24T07:59:35.701989+0000","last_became_active":"2023-11-24T07:59:34.701869+0000","last_became_peered":"2023-11-24T07:59:34.701869+0000","last_unstale":"2023-11-24T07:59:35.701989+0000","last_undegraded":"2023-11-24T07:59:35.701989+0000","last_fullsized":"2023-11-24T07:59:35.701989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.708979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698516+0000","last_change":"2023-11-24T07:59:33.700062+0000","last_active":"2023-11-24T07:59:35.698516+0000","last_peered":"2023-11-24T07:59:35.698516+0000","last_clean":"2023-11-24T07:59:35.698516+0000","last_became_active":"2023-11-24T07:59:33.699653+0000","last_became_peered":"2023-11-24T07:59:33.699653+0000","last_unstale":"2023-11-24T07:59:35.698516+0000","last_undegraded":"2023-11-24T07:59:35.698516+0000","last_fullsized":"2023-11-24T07:59:35.698516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316904+0000","last_change":"2023-11-24T07:59:34.695614+0000","last_active":"2023-11-24T07:59:36.316904+0000","last_peered":"2023-11-24T07:59:36.316904+0000","last_clean":"2023-11-24T07:59:36.316904+0000","last_became_active":"2023-11-24T07:59:34.694400+0000","last_became_peered":"2023-11-24T07:59:34.694400+0000","last_unstale":"2023-11-24T07:59:36.316904+0000","last_undegraded":"2023-11-24T07:59:36.316904+0000","last_fullsized":"2023-11-24T07:59:36.316904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:21.250798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317301+0000","last_change":"2023-11-24T07:59:33.695933+0000","last_active":"2023-11-24T07:59:36.317301+0000","last_peered":"2023-11-24T07:59:36.317301+0000","last_clean":"2023-11-24T07:59:36.317301+0000","last_became_active":"2023-11-24T07:59:33.695629+0000","last_became_peered":"2023-11-24T07:59:33.695629+0000","last_unstale":"2023-11-24T07:59:36.317301+0000","last_undegraded":"2023-11-24T07:59:36.317301+0000","last_fullsized":"2023-11-24T07:59:36.317301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698444+0000","last_change":"2023-11-24T07:59:34.712837+0000","last_active":"2023-11-24T07:59:35.698444+0000","last_peered":"2023-11-24T07:59:35.698444+0000","last_clean":"2023-11-24T07:59:35.698444+0000","last_became_active":"2023-11-24T07:59:34.712585+0000","last_became_peered":"2023-11-24T07:59:34.712585+0000","last_unstale":"2023-11-24T07:59:35.698444+0000","last_undegraded":"2023-11-24T07:59:35.698444+0000","last_fullsized":"2023-11-24T07:59:35.698444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:31.838023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316899+0000","last_change":"2023-11-24T07:59:33.699376+0000","last_active":"2023-11-24T07:59:36.316899+0000","last_peered":"2023-11-24T07:59:36.316899+0000","last_clean":"2023-11-24T07:59:36.316899+0000","last_became_active":"2023-11-24T07:59:33.699045+0000","last_became_peered":"2023-11-24T07:59:33.699045+0000","last_unstale":"2023-11-24T07:59:36.316899+0000","last_undegraded":"2023-11-24T07:59:36.316899+0000","last_fullsized":"2023-11-24T07:59:36.316899+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316845+0000","last_change":"2023-11-24T07:59:34.702610+0000","last_active":"2023-11-24T07:59:36.316845+0000","last_peered":"2023-11-24T07:59:36.316845+0000","last_clean":"2023-11-24T07:59:36.316845+0000","last_became_active":"2023-11-24T07:59:34.702263+0000","last_became_peered":"2023-11-24T07:59:34.702263+0000","last_unstale":"2023-11-24T07:59:36.316845+0000","last_undegraded":"2023-11-24T07:59:36.316845+0000","last_fullsized":"2023-11-24T07:59:36.316845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:31.838023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698231+0000","last_change":"2023-11-24T07:59:33.702517+0000","last_active":"2023-11-24T07:59:35.698231+0000","last_peered":"2023-11-24T07:59:35.698231+0000","last_clean":"2023-11-24T07:59:35.698231+0000","last_became_active":"2023-11-24T07:59:33.702216+0000","last_became_peered":"2023-11-24T07:59:33.702216+0000","last_unstale":"2023-11-24T07:59:35.698231+0000","last_undegraded":"2023-11-24T07:59:35.698231+0000","last_fullsized":"2023-11-24T07:59:35.698231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:57.344256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701952+0000","last_change":"2023-11-24T07:59:34.704646+0000","last_active":"2023-11-24T07:59:35.701952+0000","last_peered":"2023-11-24T07:59:35.701952+0000","last_clean":"2023-11-24T07:59:35.701952+0000","last_became_active":"2023-11-24T07:59:34.704140+0000","last_became_peered":"2023-11-24T07:59:34.704140+0000","last_unstale":"2023-11-24T07:59:35.701952+0000","last_undegraded":"2023-11-24T07:59:35.701952+0000","last_fullsized":"2023-11-24T07:59:35.701952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:38.054671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698038+0000","last_change":"2023-11-24T07:59:33.697712+0000","last_active":"2023-11-24T07:59:35.698038+0000","last_peered":"2023-11-24T07:59:35.698038+0000","last_clean":"2023-11-24T07:59:35.698038+0000","last_became_active":"2023-11-24T07:59:33.697416+0000","last_became_peered":"2023-11-24T07:59:33.697416+0000","last_unstale":"2023-11-24T07:59:35.698038+0000","last_undegraded":"2023-11-24T07:59:35.698038+0000","last_fullsized":"2023-11-24T07:59:35.698038+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:22.490220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317458+0000","last_change":"2023-11-24T07:59:34.698174+0000","last_active":"2023-11-24T07:59:36.317458+0000","last_peered":"2023-11-24T07:59:36.317458+0000","last_clean":"2023-11-24T07:59:36.317458+0000","last_became_active":"2023-11-24T07:59:34.697893+0000","last_became_peered":"2023-11-24T07:59:34.697893+0000","last_unstale":"2023-11-24T07:59:36.317458+0000","last_undegraded":"2023-11-24T07:59:36.317458+0000","last_fullsized":"2023-11-24T07:59:36.317458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317469+0000","last_change":"2023-11-24T07:59:33.697751+0000","last_active":"2023-11-24T07:59:36.317469+0000","last_peered":"2023-11-24T07:59:36.317469+0000","last_clean":"2023-11-24T07:59:36.317469+0000","last_became_active":"2023-11-24T07:59:33.696926+0000","last_became_peered":"2023-11-24T07:59:33.696926+0000","last_unstale":"2023-11-24T07:59:36.317469+0000","last_undegraded":"2023-11-24T07:59:36.317469+0000","last_fullsized":"2023-11-24T07:59:36.317469+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:03.318191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702361+0000","last_change":"2023-11-24T07:59:34.700908+0000","last_active":"2023-11-24T07:59:35.702361+0000","last_peered":"2023-11-24T07:59:35.702361+0000","last_clean":"2023-11-24T07:59:35.702361+0000","last_became_active":"2023-11-24T07:59:34.700430+0000","last_became_peered":"2023-11-24T07:59:34.700430+0000","last_unstale":"2023-11-24T07:59:35.702361+0000","last_undegraded":"2023-11-24T07:59:35.702361+0000","last_fullsized":"2023-11-24T07:59:35.702361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:59.781689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317511+0000","last_change":"2023-11-24T07:59:33.695522+0000","last_active":"2023-11-24T07:59:36.317511+0000","last_peered":"2023-11-24T07:59:36.317511+0000","last_clean":"2023-11-24T07:59:36.317511+0000","last_became_active":"2023-11-24T07:59:33.694756+0000","last_became_peered":"2023-11-24T07:59:33.694756+0000","last_unstale":"2023-11-24T07:59:36.317511+0000","last_undegraded":"2023-11-24T07:59:36.317511+0000","last_fullsized":"2023-11-24T07:59:36.317511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:56.663996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317243+0000","last_change":"2023-11-24T07:59:34.707523+0000","last_active":"2023-11-24T07:59:36.317243+0000","last_peered":"2023-11-24T07:59:36.317243+0000","last_clean":"2023-11-24T07:59:36.317243+0000","last_became_active":"2023-11-24T07:59:34.706844+0000","last_became_peered":"2023-11-24T07:59:34.706844+0000","last_unstale":"2023-11-24T07:59:36.317243+0000","last_undegraded":"2023-11-24T07:59:36.317243+0000","last_fullsized":"2023-11-24T07:59:36.317243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:54.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317196+0000","last_change":"2023-11-24T07:59:33.694599+0000","last_active":"2023-11-24T07:59:36.317196+0000","last_peered":"2023-11-24T07:59:36.317196+0000","last_clean":"2023-11-24T07:59:36.317196+0000","last_became_active":"2023-11-24T07:59:33.693972+0000","last_became_peered":"2023-11-24T07:59:33.693972+0000","last_unstale":"2023-11-24T07:59:36.317196+0000","last_undegraded":"2023-11-24T07:59:36.317196+0000","last_fullsized":"2023-11-24T07:59:36.317196+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:39.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316513+0000","last_change":"2023-11-24T07:59:34.708238+0000","last_active":"2023-11-24T07:59:36.316513+0000","last_peered":"2023-11-24T07:59:36.316513+0000","last_clean":"2023-11-24T07:59:36.316513+0000","last_became_active":"2023-11-24T07:59:34.707944+0000","last_became_peered":"2023-11-24T07:59:34.707944+0000","last_unstale":"2023-11-24T07:59:36.316513+0000","last_undegraded":"2023-11-24T07:59:36.316513+0000","last_fullsized":"2023-11-24T07:59:36.316513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:11.165477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317502+0000","last_change":"2023-11-24T07:59:33.696954+0000","last_active":"2023-11-24T07:59:36.317502+0000","last_peered":"2023-11-24T07:59:36.317502+0000","last_clean":"2023-11-24T07:59:36.317502+0000","last_became_active":"2023-11-24T07:59:33.696594+0000","last_became_peered":"2023-11-24T07:59:33.696594+0000","last_unstale":"2023-11-24T07:59:36.317502+0000","last_undegraded":"2023-11-24T07:59:36.317502+0000","last_fullsized":"2023-11-24T07:59:36.317502+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:12:54.321416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316210+0000","last_change":"2023-11-24T07:59:34.701913+0000","last_active":"2023-11-24T07:59:36.316210+0000","last_peered":"2023-11-24T07:59:36.316210+0000","last_clean":"2023-11-24T07:59:36.316210+0000","last_became_active":"2023-11-24T07:59:34.701254+0000","last_became_peered":"2023-11-24T07:59:34.701254+0000","last_unstale":"2023-11-24T07:59:36.316210+0000","last_undegraded":"2023-11-24T07:59:36.316210+0000","last_fullsized":"2023-11-24T07:59:36.316210+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:18:56.985255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316226+0000","last_change":"2023-11-24T07:59:33.700636+0000","last_active":"2023-11-24T07:59:36.316226+0000","last_peered":"2023-11-24T07:59:36.316226+0000","last_clean":"2023-11-24T07:59:36.316226+0000","last_became_active":"2023-11-24T07:59:33.700236+0000","last_became_peered":"2023-11-24T07:59:33.700236+0000","last_unstale":"2023-11-24T07:59:36.316226+0000","last_undegraded":"2023-11-24T07:59:36.316226+0000","last_fullsized":"2023-11-24T07:59:36.316226+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316652+0000","last_change":"2023-11-24T07:59:34.713518+0000","last_active":"2023-11-24T07:59:36.316652+0000","last_peered":"2023-11-24T07:59:36.316652+0000","last_clean":"2023-11-24T07:59:36.316652+0000","last_became_active":"2023-11-24T07:59:34.713162+0000","last_became_peered":"2023-11-24T07:59:34.713162+0000","last_unstale":"2023-11-24T07:59:36.316652+0000","last_undegraded":"2023-11-24T07:59:36.316652+0000","last_fullsized":"2023-11-24T07:59:36.316652+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.480402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702504+0000","last_change":"2023-11-24T07:59:33.700732+0000","last_active":"2023-11-24T07:59:35.702504+0000","last_peered":"2023-11-24T07:59:35.702504+0000","last_clean":"2023-11-24T07:59:35.702504+0000","last_became_active":"2023-11-24T07:59:33.700454+0000","last_became_peered":"2023-11-24T07:59:33.700454+0000","last_unstale":"2023-11-24T07:59:35.702504+0000","last_undegraded":"2023-11-24T07:59:35.702504+0000","last_fullsized":"2023-11-24T07:59:35.702504+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:39.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316163+0000","last_change":"2023-11-24T07:59:34.709505+0000","last_active":"2023-11-24T07:59:36.316163+0000","last_peered":"2023-11-24T07:59:36.316163+0000","last_clean":"2023-11-24T07:59:36.316163+0000","last_became_active":"2023-11-24T07:59:34.706917+0000","last_became_peered":"2023-11-24T07:59:34.706917+0000","last_unstale":"2023-11-24T07:59:36.316163+0000","last_undegraded":"2023-11-24T07:59:36.316163+0000","last_fullsized":"2023-11-24T07:59:36.316163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:58:50.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316917+0000","last_change":"2023-11-24T07:59:33.700333+0000","last_active":"2023-11-24T07:59:36.316917+0000","last_peered":"2023-11-24T07:59:36.316917+0000","last_clean":"2023-11-24T07:59:36.316917+0000","last_became_active":"2023-11-24T07:59:33.699990+0000","last_became_peered":"2023-11-24T07:59:33.699990+0000","last_unstale":"2023-11-24T07:59:36.316917+0000","last_undegraded":"2023-11-24T07:59:36.316917+0000","last_fullsized":"2023-11-24T07:59:36.316917+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:10.160866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316889+0000","last_change":"2023-11-24T07:59:34.706961+0000","last_active":"2023-11-24T07:59:36.316889+0000","last_peered":"2023-11-24T07:59:36.316889+0000","last_clean":"2023-11-24T07:59:36.316889+0000","last_became_active":"2023-11-24T07:59:34.706269+0000","last_became_peered":"2023-11-24T07:59:34.706269+0000","last_unstale":"2023-11-24T07:59:36.316889+0000","last_undegraded":"2023-11-24T07:59:36.316889+0000","last_fullsized":"2023-11-24T07:59:36.316889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:18.811326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701831+0000","last_change":"2023-11-24T07:59:33.703879+0000","last_active":"2023-11-24T07:59:35.701831+0000","last_peered":"2023-11-24T07:59:35.701831+0000","last_clean":"2023-11-24T07:59:35.701831+0000","last_became_active":"2023-11-24T07:59:33.703565+0000","last_became_peered":"2023-11-24T07:59:33.703565+0000","last_unstale":"2023-11-24T07:59:35.701831+0000","last_undegraded":"2023-11-24T07:59:35.701831+0000","last_fullsized":"2023-11-24T07:59:35.701831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:40.595331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316678+0000","last_change":"2023-11-24T07:59:34.698078+0000","last_active":"2023-11-24T07:59:36.316678+0000","last_peered":"2023-11-24T07:59:36.316678+0000","last_clean":"2023-11-24T07:59:36.316678+0000","last_became_active":"2023-11-24T07:59:34.697740+0000","last_became_peered":"2023-11-24T07:59:34.697740+0000","last_unstale":"2023-11-24T07:59:36.316678+0000","last_undegraded":"2023-11-24T07:59:36.316678+0000","last_fullsized":"2023-11-24T07:59:36.316678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:04.115346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317347+0000","last_change":"2023-11-24T07:59:33.697566+0000","last_active":"2023-11-24T07:59:36.317347+0000","last_peered":"2023-11-24T07:59:36.317347+0000","last_clean":"2023-11-24T07:59:36.317347+0000","last_became_active":"2023-11-24T07:59:33.697280+0000","last_became_peered":"2023-11-24T07:59:33.697280+0000","last_unstale":"2023-11-24T07:59:36.317347+0000","last_undegraded":"2023-11-24T07:59:36.317347+0000","last_fullsized":"2023-11-24T07:59:36.317347+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:22.490220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702209+0000","last_change":"2023-11-24T07:59:34.716265+0000","last_active":"2023-11-24T07:59:35.702209+0000","last_peered":"2023-11-24T07:59:35.702209+0000","last_clean":"2023-11-24T07:59:35.702209+0000","last_became_active":"2023-11-24T07:59:34.715970+0000","last_became_peered":"2023-11-24T07:59:34.715970+0000","last_unstale":"2023-11-24T07:59:35.702209+0000","last_undegraded":"2023-11-24T07:59:35.702209+0000","last_fullsized":"2023-11-24T07:59:35.702209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:10.747185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316677+0000","last_change":"2023-11-24T07:59:33.697890+0000","last_active":"2023-11-24T07:59:36.316677+0000","last_peered":"2023-11-24T07:59:36.316677+0000","last_clean":"2023-11-24T07:59:36.316677+0000","last_became_active":"2023-11-24T07:59:33.697587+0000","last_became_peered":"2023-11-24T07:59:33.697587+0000","last_unstale":"2023-11-24T07:59:36.316677+0000","last_undegraded":"2023-11-24T07:59:36.316677+0000","last_fullsized":"2023-11-24T07:59:36.316677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:37.050060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316061+0000","last_change":"2023-11-24T07:59:34.717080+0000","last_active":"2023-11-24T07:59:36.316061+0000","last_peered":"2023-11-24T07:59:36.316061+0000","last_clean":"2023-11-24T07:59:36.316061+0000","last_became_active":"2023-11-24T07:59:34.716798+0000","last_became_peered":"2023-11-24T07:59:34.716798+0000","last_unstale":"2023-11-24T07:59:36.316061+0000","last_undegraded":"2023-11-24T07:59:36.316061+0000","last_fullsized":"2023-11-24T07:59:36.316061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:58.348867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317302+0000","last_change":"2023-11-24T07:59:33.700926+0000","last_active":"2023-11-24T07:59:36.317302+0000","last_peered":"2023-11-24T07:59:36.317302+0000","last_clean":"2023-11-24T07:59:36.317302+0000","last_became_active":"2023-11-24T07:59:33.700542+0000","last_became_peered":"2023-11-24T07:59:33.700542+0000","last_unstale":"2023-11-24T07:59:36.317302+0000","last_undegraded":"2023-11-24T07:59:36.317302+0000","last_fullsized":"2023-11-24T07:59:36.317302+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:57.344256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702418+0000","last_change":"2023-11-24T07:59:34.709122+0000","last_active":"2023-11-24T07:59:35.702418+0000","last_peered":"2023-11-24T07:59:35.702418+0000","last_clean":"2023-11-24T07:59:35.702418+0000","last_became_active":"2023-11-24T07:59:34.708781+0000","last_became_peered":"2023-11-24T07:59:34.708781+0000","last_unstale":"2023-11-24T07:59:35.702418+0000","last_undegraded":"2023-11-24T07:59:35.702418+0000","last_fullsized":"2023-11-24T07:59:35.702418+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:03.261346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316191+0000","last_change":"2023-11-24T07:59:33.696176+0000","last_active":"2023-11-24T07:59:36.316191+0000","last_peered":"2023-11-24T07:59:36.316191+0000","last_clean":"2023-11-24T07:59:36.316191+0000","last_became_active":"2023-11-24T07:59:33.695894+0000","last_became_peered":"2023-11-24T07:59:33.695894+0000","last_unstale":"2023-11-24T07:59:36.316191+0000","last_undegraded":"2023-11-24T07:59:36.316191+0000","last_fullsized":"2023-11-24T07:59:36.316191+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:56.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316326+0000","last_change":"2023-11-24T07:59:34.712932+0000","last_active":"2023-11-24T07:59:36.316326+0000","last_peered":"2023-11-24T07:59:36.316326+0000","last_clean":"2023-11-24T07:59:36.316326+0000","last_became_active":"2023-11-24T07:59:34.712521+0000","last_became_peered":"2023-11-24T07:59:34.712521+0000","last_unstale":"2023-11-24T07:59:36.316326+0000","last_undegraded":"2023-11-24T07:59:36.316326+0000","last_fullsized":"2023-11-24T07:59:36.316326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:23.494831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317007+0000","last_change":"2023-11-24T07:59:33.699949+0000","last_active":"2023-11-24T07:59:36.317007+0000","last_peered":"2023-11-24T07:59:36.317007+0000","last_clean":"2023-11-24T07:59:36.317007+0000","last_became_active":"2023-11-24T07:59:33.699614+0000","last_became_peered":"2023-11-24T07:59:33.699614+0000","last_unstale":"2023-11-24T07:59:36.317007+0000","last_undegraded":"2023-11-24T07:59:36.317007+0000","last_fullsized":"2023-11-24T07:59:36.317007+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:56.663996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698078+0000","last_change":"2023-11-24T07:59:34.691124+0000","last_active":"2023-11-24T07:59:35.698078+0000","last_peered":"2023-11-24T07:59:35.698078+0000","last_clean":"2023-11-24T07:59:35.698078+0000","last_became_active":"2023-11-24T07:59:34.690758+0000","last_became_peered":"2023-11-24T07:59:34.690758+0000","last_unstale":"2023-11-24T07:59:35.698078+0000","last_undegraded":"2023-11-24T07:59:35.698078+0000","last_fullsized":"2023-11-24T07:59:35.698078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:10.747185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317026+0000","last_change":"2023-11-24T07:59:33.696331+0000","last_active":"2023-11-24T07:59:36.317026+0000","last_peered":"2023-11-24T07:59:36.317026+0000","last_clean":"2023-11-24T07:59:36.317026+0000","last_became_active":"2023-11-24T07:59:33.694494+0000","last_became_peered":"2023-11-24T07:59:33.694494+0000","last_unstale":"2023-11-24T07:59:36.317026+0000","last_undegraded":"2023-11-24T07:59:36.317026+0000","last_fullsized":"2023-11-24T07:59:36.317026+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:05.919483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316744+0000","last_change":"2023-11-24T07:59:34.707779+0000","last_active":"2023-11-24T07:59:36.316744+0000","last_peered":"2023-11-24T07:59:36.316744+0000","last_clean":"2023-11-24T07:59:36.316744+0000","last_became_active":"2023-11-24T07:59:34.707540+0000","last_became_peered":"2023-11-24T07:59:34.707540+0000","last_unstale":"2023-11-24T07:59:36.316744+0000","last_undegraded":"2023-11-24T07:59:36.316744+0000","last_fullsized":"2023-11-24T07:59:36.316744+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:43.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698166+0000","last_change":"2023-11-24T07:59:33.703107+0000","last_active":"2023-11-24T07:59:35.698166+0000","last_peered":"2023-11-24T07:59:35.698166+0000","last_clean":"2023-11-24T07:59:35.698166+0000","last_became_active":"2023-11-24T07:59:33.702844+0000","last_became_peered":"2023-11-24T07:59:33.702844+0000","last_unstale":"2023-11-24T07:59:35.698166+0000","last_undegraded":"2023-11-24T07:59:35.698166+0000","last_fullsized":"2023-11-24T07:59:35.698166+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:42.136935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316637+0000","last_change":"2023-11-24T07:59:34.708726+0000","last_active":"2023-11-24T07:59:36.316637+0000","last_peered":"2023-11-24T07:59:36.316637+0000","last_clean":"2023-11-24T07:59:36.316637+0000","last_became_active":"2023-11-24T07:59:34.708433+0000","last_became_peered":"2023-11-24T07:59:34.708433+0000","last_unstale":"2023-11-24T07:59:36.316637+0000","last_undegraded":"2023-11-24T07:59:36.316637+0000","last_fullsized":"2023-11-24T07:59:36.316637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.708979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702200+0000","last_change":"2023-11-24T07:59:33.695863+0000","last_active":"2023-11-24T07:59:35.702200+0000","last_peered":"2023-11-24T07:59:35.702200+0000","last_clean":"2023-11-24T07:59:35.702200+0000","last_became_active":"2023-11-24T07:59:33.695557+0000","last_became_peered":"2023-11-24T07:59:33.695557+0000","last_unstale":"2023-11-24T07:59:35.702200+0000","last_undegraded":"2023-11-24T07:59:35.702200+0000","last_fullsized":"2023-11-24T07:59:35.702200+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:12.814684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-24T07:59:34.688534+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:59:34.688534+0000","last_peered":"2023-11-24T07:59:34.688534+0000","last_clean":"2023-11-24T07:59:34.688534+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:59:34.688534+0000","last_undegraded":"2023-11-24T07:59:34.688534+0000","last_fullsized":"2023-11-24T07:59:34.688534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":729231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.62}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.70899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":139389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.71299999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.623},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":140407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.751}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.316}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.37},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.433}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.49299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":132543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.377}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.62},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":134107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.751},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.78800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":138877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.621},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.55000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":139445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.748}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:38.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.006+0000 7fe58e5b7700 1 Processor -- start 2023-11-24T07:59:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.007+0000 7fe58e5b7700 1 -- start start 2023-11-24T07:59:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.007+0000 7fe58e5b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe58806c710 0x7fe588165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.007+0000 7fe58e5b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880af950 con 0x7fe5881446a0 2023-11-24T07:59:39.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.007+0000 7fe58e5b7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880afc30 con 0x7fe5880adf00 2023-11-24T07:59:39.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.007+0000 7fe58e5b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe588144a80 con 0x7fe58806c710 2023-11-24T07:59:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.007+0000 7fe587fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5881446a0 0x7fe5880b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60152/0 (socket says 172.21.15.32:60152) 2023-11-24T07:59:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe587fff700 1 -- 172.21.15.32:0/903485263 learned_addr learned my addr 172.21.15.32:0/903485263 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe58cb54700 1 --2- 172.21.15.32:0/903485263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe58806c710 0x7fe588165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264100377 0 0) 0x7fe5880af950 con 0x7fe5881446a0 2023-11-24T07:59:39.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5740035b0 con 0x7fe5881446a0 2023-11-24T07:59:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549507633 0 0) 0x7fe5880afc30 con 0x7fe5880adf00 2023-11-24T07:59:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5880af950 con 0x7fe5880adf00 2023-11-24T07:59:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219614731 0 0) 0x7fe5740035b0 con 0x7fe5881446a0 2023-11-24T07:59:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe58cb54700 1 -- 172.21.15.32:0/903485263 >> v1:172.21.15.123:6789/0 conn(0x7fe5880adf00 legacy=0x7fe58806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe58cb54700 1 -- 172.21.15.32:0/903485263 >> v1:172.21.15.32:6789/0 conn(0x7fe5881446a0 legacy=0x7fe5880b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe58cb54700 1 -- 172.21.15.32:0/903485263 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5880aff10 con 0x7fe58806c710 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe58cb54700 1 --2- 172.21.15.32:0/903485263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe58806c710 0x7fe588165b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe580009dc0 tx=0x7fe58000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc8a704c97836a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000e070 con 0x7fe58806c710 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580003a20 con 0x7fe58806c710 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.008+0000 7fe586ffd700 1 -- 172.21.15.32:0/903485263 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578001670 con 0x7fe58806c710 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 -- 172.21.15.32:0/903485263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe58806c710 msgr2=0x7fe588165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/903485263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe58806c710 0x7fe588165b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe580009dc0 tx=0x7fe58000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:39.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 -- 172.21.15.32:0/903485263 shutdown_connections 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/903485263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe58806c710 0x7fe588165b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 -- 172.21.15.32:0/903485263 >> 172.21.15.32:0/903485263 conn(0x7fe588071b90 msgr2=0x7fe588071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 -- 172.21.15.32:0/903485263 shutdown_connections 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.009+0000 7fe58e5b7700 1 -- 172.21.15.32:0/903485263 wait complete. 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.010+0000 7fe58e5b7700 1 Processor -- start 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.010+0000 7fe58e5b7700 1 -- start start 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58e5b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe58806c710 0x7fe588143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58e5b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 0x7fe5880d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58e5b7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5881446a0 0x7fe58815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58e5b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58815ab30 con 0x7fe5880adf00 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58e5b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58813f6f0 con 0x7fe58806c710 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58e5b7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58813f130 con 0x7fe5881446a0 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 0x7fe5880d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 0x7fe5880d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58448/0 (socket says 172.21.15.32:58448) 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe5877fe700 1 -- 172.21.15.32:0/3632524672 learned_addr learned my addr 172.21.15.32:0/3632524672 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.011+0000 7fe58cb54700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5881446a0 0x7fe58815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe587fff700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe58806c710 0x7fe588143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe5877fe700 1 -- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5881446a0 msgr2=0x7fe58815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe5877fe700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5881446a0 0x7fe58815ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe5877fe700 1 -- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe58806c710 msgr2=0x7fe588143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe5877fe700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe58806c710 0x7fe588143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe5877fe700 1 -- 172.21.15.32:0/3632524672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009a70 con 0x7fe5880adf00 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe5877fe700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 0x7fe5880d00a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c460 tx=0x7fe57000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700052d0 con 0x7fe5880adf00 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570005450 con 0x7fe5880adf00 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700038c0 con 0x7fe5880adf00 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58813f410 con 0x7fe5880adf00 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.012+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5880cea20 con 0x7fe5880adf00 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.013+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5880753f0 con 0x7fe5880adf00 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.014+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe570003a20 con 0x7fe5880adf00 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.014+0000 7fe584ff9700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe56806aed0 0x7fe56806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.014+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5700889c0 con 0x7fe5880adf00 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.014+0000 7fe587fff700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe56806aed0 0x7fe56806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.015+0000 7fe587fff700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe56806aed0 0x7fe56806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe578007fd0 tx=0x7fe578007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.019+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe570053b00 con 0x7fe5880adf00 2023-11-24T07:59:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.217+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5880b04d0 con 0x7fe56806aed0 2023-11-24T07:59:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.225+0000 7fe584ff9700 1 -- 172.21.15.32:0/3632524672 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288654 (secure 0 0 0) 0x7fe578007620 con 0x7fe56806aed0 2023-11-24T07:59:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe56806aed0 msgr2=0x7fe56806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe56806aed0 0x7fe56806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe578007fd0 tx=0x7fe578007a60 comp rx=0 tx=0).stop 2023-11-24T07:59:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 msgr2=0x7fe5880d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 0x7fe5880d00a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c460 tx=0x7fe57000c820 comp rx=0 tx=0).stop 2023-11-24T07:59:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 shutdown_connections 2023-11-24T07:59:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe56806aed0 0x7fe56806d390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5881446a0 0x7fe58815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe58806c710 0x7fe588143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 --2- 172.21.15.32:0/3632524672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5880adf00 0x7fe5880d00a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.228+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 >> 172.21.15.32:0/3632524672 conn(0x7fe588071b90 msgr2=0x7fe588073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.229+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 shutdown_connections 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.229+0000 7fe58e5b7700 1 -- 172.21.15.32:0/3632524672 wait complete. 2023-11-24T07:59:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T07:59:39.243 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-11-24T07:59:38.544987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220184,"kb_used_data":4004,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754536,"statfs":{"total":773094113280,"available":772868644864,"internally_reserved":0,"allocated":4100096,"data_stored":2276840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.866103"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702699+0000","last_change":"2023-11-24T07:59:34.696902+0000","last_active":"2023-11-24T07:59:35.702699+0000","last_peered":"2023-11-24T07:59:35.702699+0000","last_clean":"2023-11-24T07:59:35.702699+0000","last_became_active":"2023-11-24T07:59:34.696588+0000","last_became_peered":"2023-11-24T07:59:34.696588+0000","last_unstale":"2023-11-24T07:59:35.702699+0000","last_undegraded":"2023-11-24T07:59:35.702699+0000","last_fullsized":"2023-11-24T07:59:35.702699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:58.348867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698786+0000","last_change":"2023-11-24T07:59:34.708947+0000","last_active":"2023-11-24T07:59:35.698786+0000","last_peered":"2023-11-24T07:59:35.698786+0000","last_clean":"2023-11-24T07:59:35.698786+0000","last_became_active":"2023-11-24T07:59:34.708625+0000","last_became_peered":"2023-11-24T07:59:34.708625+0000","last_unstale":"2023-11-24T07:59:35.698786+0000","last_undegraded":"2023-11-24T07:59:35.698786+0000","last_fullsized":"2023-11-24T07:59:35.698786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:25.675685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701945+0000","last_change":"2023-11-24T07:59:34.702426+0000","last_active":"2023-11-24T07:59:35.701945+0000","last_peered":"2023-11-24T07:59:35.701945+0000","last_clean":"2023-11-24T07:59:35.701945+0000","last_became_active":"2023-11-24T07:59:34.702161+0000","last_became_peered":"2023-11-24T07:59:34.702161+0000","last_unstale":"2023-11-24T07:59:35.701945+0000","last_undegraded":"2023-11-24T07:59:35.701945+0000","last_fullsized":"2023-11-24T07:59:35.701945+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317054+0000","last_change":"2023-11-24T07:59:34.707996+0000","last_active":"2023-11-24T07:59:36.317054+0000","last_peered":"2023-11-24T07:59:36.317054+0000","last_clean":"2023-11-24T07:59:36.317054+0000","last_became_active":"2023-11-24T07:59:34.707720+0000","last_became_peered":"2023-11-24T07:59:34.707720+0000","last_unstale":"2023-11-24T07:59:36.317054+0000","last_undegraded":"2023-11-24T07:59:36.317054+0000","last_fullsized":"2023-11-24T07:59:36.317054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:09.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316356+0000","last_change":"2023-11-24T07:59:34.709389+0000","last_active":"2023-11-24T07:59:36.316356+0000","last_peered":"2023-11-24T07:59:36.316356+0000","last_clean":"2023-11-24T07:59:36.316356+0000","last_became_active":"2023-11-24T07:59:34.708883+0000","last_became_peered":"2023-11-24T07:59:34.708883+0000","last_unstale":"2023-11-24T07:59:36.316356+0000","last_undegraded":"2023-11-24T07:59:36.316356+0000","last_fullsized":"2023-11-24T07:59:36.316356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:40.366654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698537+0000","last_change":"2023-11-24T07:59:34.695237+0000","last_active":"2023-11-24T07:59:35.698537+0000","last_peered":"2023-11-24T07:59:35.698537+0000","last_clean":"2023-11-24T07:59:35.698537+0000","last_became_active":"2023-11-24T07:59:34.694926+0000","last_became_peered":"2023-11-24T07:59:34.694926+0000","last_unstale":"2023-11-24T07:59:35.698537+0000","last_undegraded":"2023-11-24T07:59:35.698537+0000","last_fullsized":"2023-11-24T07:59:35.698537+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:48.172772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316785+0000","last_change":"2023-11-24T07:59:34.706621+0000","last_active":"2023-11-24T07:59:36.316785+0000","last_peered":"2023-11-24T07:59:36.316785+0000","last_clean":"2023-11-24T07:59:36.316785+0000","last_became_active":"2023-11-24T07:59:34.706364+0000","last_became_peered":"2023-11-24T07:59:34.706364+0000","last_unstale":"2023-11-24T07:59:36.316785+0000","last_undegraded":"2023-11-24T07:59:36.316785+0000","last_fullsized":"2023-11-24T07:59:36.316785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:10.747185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316747+0000","last_change":"2023-11-24T07:59:34.706447+0000","last_active":"2023-11-24T07:59:36.316747+0000","last_peered":"2023-11-24T07:59:36.316747+0000","last_clean":"2023-11-24T07:59:36.316747+0000","last_became_active":"2023-11-24T07:59:34.705935+0000","last_became_peered":"2023-11-24T07:59:34.705935+0000","last_unstale":"2023-11-24T07:59:36.316747+0000","last_undegraded":"2023-11-24T07:59:36.316747+0000","last_fullsized":"2023-11-24T07:59:36.316747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:01.322460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317328+0000","last_change":"2023-11-24T07:59:34.716245+0000","last_active":"2023-11-24T07:59:36.317328+0000","last_peered":"2023-11-24T07:59:36.317328+0000","last_clean":"2023-11-24T07:59:36.317328+0000","last_became_active":"2023-11-24T07:59:34.715942+0000","last_became_peered":"2023-11-24T07:59:34.715942+0000","last_unstale":"2023-11-24T07:59:36.317328+0000","last_undegraded":"2023-11-24T07:59:36.317328+0000","last_fullsized":"2023-11-24T07:59:36.317328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:54.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317198+0000","last_change":"2023-11-24T07:59:34.709403+0000","last_active":"2023-11-24T07:59:36.317198+0000","last_peered":"2023-11-24T07:59:36.317198+0000","last_clean":"2023-11-24T07:59:36.317198+0000","last_became_active":"2023-11-24T07:59:34.709097+0000","last_became_peered":"2023-11-24T07:59:34.709097+0000","last_unstale":"2023-11-24T07:59:36.317198+0000","last_undegraded":"2023-11-24T07:59:36.317198+0000","last_fullsized":"2023-11-24T07:59:36.317198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:41.599941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317009+0000","last_change":"2023-11-24T07:59:34.702388+0000","last_active":"2023-11-24T07:59:36.317009+0000","last_peered":"2023-11-24T07:59:36.317009+0000","last_clean":"2023-11-24T07:59:36.317009+0000","last_became_active":"2023-11-24T07:59:34.702105+0000","last_became_peered":"2023-11-24T07:59:34.702105+0000","last_unstale":"2023-11-24T07:59:36.317009+0000","last_undegraded":"2023-11-24T07:59:36.317009+0000","last_fullsized":"2023-11-24T07:59:36.317009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:48.172772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702541+0000","last_change":"2023-11-24T07:59:34.700871+0000","last_active":"2023-11-24T07:59:35.702541+0000","last_peered":"2023-11-24T07:59:35.702541+0000","last_clean":"2023-11-24T07:59:35.702541+0000","last_became_active":"2023-11-24T07:59:34.700241+0000","last_became_peered":"2023-11-24T07:59:34.700241+0000","last_unstale":"2023-11-24T07:59:35.702541+0000","last_undegraded":"2023-11-24T07:59:35.702541+0000","last_fullsized":"2023-11-24T07:59:35.702541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:18.983207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316621+0000","last_change":"2023-11-24T07:59:34.703008+0000","last_active":"2023-11-24T07:59:36.316621+0000","last_peered":"2023-11-24T07:59:36.316621+0000","last_clean":"2023-11-24T07:59:36.316621+0000","last_became_active":"2023-11-24T07:59:34.702674+0000","last_became_peered":"2023-11-24T07:59:34.702674+0000","last_unstale":"2023-11-24T07:59:36.316621+0000","last_undegraded":"2023-11-24T07:59:36.316621+0000","last_fullsized":"2023-11-24T07:59:36.316621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:13.819295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698568+0000","last_change":"2023-11-24T07:59:34.710511+0000","last_active":"2023-11-24T07:59:35.698568+0000","last_peered":"2023-11-24T07:59:35.698568+0000","last_clean":"2023-11-24T07:59:35.698568+0000","last_became_active":"2023-11-24T07:59:34.709859+0000","last_became_peered":"2023-11-24T07:59:34.709859+0000","last_unstale":"2023-11-24T07:59:35.698568+0000","last_undegraded":"2023-11-24T07:59:35.698568+0000","last_fullsized":"2023-11-24T07:59:35.698568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:18.811326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702407+0000","last_change":"2023-11-24T07:59:34.706291+0000","last_active":"2023-11-24T07:59:35.702407+0000","last_peered":"2023-11-24T07:59:35.702407+0000","last_clean":"2023-11-24T07:59:35.702407+0000","last_became_active":"2023-11-24T07:59:34.705978+0000","last_became_peered":"2023-11-24T07:59:34.705978+0000","last_unstale":"2023-11-24T07:59:35.702407+0000","last_undegraded":"2023-11-24T07:59:35.702407+0000","last_fullsized":"2023-11-24T07:59:35.702407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:31.838023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316591+0000","last_change":"2023-11-24T07:59:34.710684+0000","last_active":"2023-11-24T07:59:36.316591+0000","last_peered":"2023-11-24T07:59:36.316591+0000","last_clean":"2023-11-24T07:59:36.316591+0000","last_became_active":"2023-11-24T07:59:34.710435+0000","last_became_peered":"2023-11-24T07:59:34.710435+0000","last_unstale":"2023-11-24T07:59:36.316591+0000","last_undegraded":"2023-11-24T07:59:36.316591+0000","last_fullsized":"2023-11-24T07:59:36.316591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316984+0000","last_change":"2023-11-24T07:59:34.707171+0000","last_active":"2023-11-24T07:59:36.316984+0000","last_peered":"2023-11-24T07:59:36.316984+0000","last_clean":"2023-11-24T07:59:36.316984+0000","last_became_active":"2023-11-24T07:59:34.706918+0000","last_became_peered":"2023-11-24T07:59:34.706918+0000","last_unstale":"2023-11-24T07:59:36.316984+0000","last_undegraded":"2023-11-24T07:59:36.316984+0000","last_fullsized":"2023-11-24T07:59:36.316984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:18.983207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698781+0000","last_change":"2023-11-24T07:59:34.706692+0000","last_active":"2023-11-24T07:59:35.698781+0000","last_peered":"2023-11-24T07:59:35.698781+0000","last_clean":"2023-11-24T07:59:35.698781+0000","last_became_active":"2023-11-24T07:59:34.706277+0000","last_became_peered":"2023-11-24T07:59:34.706277+0000","last_unstale":"2023-11-24T07:59:35.698781+0000","last_undegraded":"2023-11-24T07:59:35.698781+0000","last_fullsized":"2023-11-24T07:59:35.698781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:42.122406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698488+0000","last_change":"2023-11-24T07:59:34.693674+0000","last_active":"2023-11-24T07:59:35.698488+0000","last_peered":"2023-11-24T07:59:35.698488+0000","last_clean":"2023-11-24T07:59:35.698488+0000","last_became_active":"2023-11-24T07:59:34.693387+0000","last_became_peered":"2023-11-24T07:59:34.693387+0000","last_unstale":"2023-11-24T07:59:35.698488+0000","last_undegraded":"2023-11-24T07:59:35.698488+0000","last_fullsized":"2023-11-24T07:59:35.698488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:18.983207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316679+0000","last_change":"2023-11-24T07:59:34.707725+0000","last_active":"2023-11-24T07:59:36.316679+0000","last_peered":"2023-11-24T07:59:36.316679+0000","last_clean":"2023-11-24T07:59:36.316679+0000","last_became_active":"2023-11-24T07:59:34.707395+0000","last_became_peered":"2023-11-24T07:59:34.707395+0000","last_unstale":"2023-11-24T07:59:36.316679+0000","last_undegraded":"2023-11-24T07:59:36.316679+0000","last_fullsized":"2023-11-24T07:59:36.316679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:06.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317074+0000","last_change":"2023-11-24T07:59:34.712358+0000","last_active":"2023-11-24T07:59:36.317074+0000","last_peered":"2023-11-24T07:59:36.317074+0000","last_clean":"2023-11-24T07:59:36.317074+0000","last_became_active":"2023-11-24T07:59:34.712098+0000","last_became_peered":"2023-11-24T07:59:34.712098+0000","last_unstale":"2023-11-24T07:59:36.317074+0000","last_undegraded":"2023-11-24T07:59:36.317074+0000","last_fullsized":"2023-11-24T07:59:36.317074+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:09.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317331+0000","last_change":"2023-11-24T07:59:34.710729+0000","last_active":"2023-11-24T07:59:36.317331+0000","last_peered":"2023-11-24T07:59:36.317331+0000","last_clean":"2023-11-24T07:59:36.317331+0000","last_became_active":"2023-11-24T07:59:34.710425+0000","last_became_peered":"2023-11-24T07:59:34.710425+0000","last_unstale":"2023-11-24T07:59:36.317331+0000","last_undegraded":"2023-11-24T07:59:36.317331+0000","last_fullsized":"2023-11-24T07:59:36.317331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:21.250798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.315933+0000","last_change":"2023-11-24T07:59:34.705414+0000","last_active":"2023-11-24T07:59:36.315933+0000","last_peered":"2023-11-24T07:59:36.315933+0000","last_clean":"2023-11-24T07:59:36.315933+0000","last_became_active":"2023-11-24T07:59:34.704888+0000","last_became_peered":"2023-11-24T07:59:34.704888+0000","last_unstale":"2023-11-24T07:59:36.315933+0000","last_undegraded":"2023-11-24T07:59:36.315933+0000","last_fullsized":"2023-11-24T07:59:36.315933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:57.668606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701794+0000","last_change":"2023-11-24T07:59:34.695742+0000","last_active":"2023-11-24T07:59:35.701794+0000","last_peered":"2023-11-24T07:59:35.701794+0000","last_clean":"2023-11-24T07:59:35.701794+0000","last_became_active":"2023-11-24T07:59:34.695436+0000","last_became_peered":"2023-11-24T07:59:34.695436+0000","last_unstale":"2023-11-24T07:59:35.701794+0000","last_undegraded":"2023-11-24T07:59:35.701794+0000","last_fullsized":"2023-11-24T07:59:35.701794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317356+0000","last_change":"2023-11-24T07:59:34.715895+0000","last_active":"2023-11-24T07:59:36.317356+0000","last_peered":"2023-11-24T07:59:36.317356+0000","last_clean":"2023-11-24T07:59:36.317356+0000","last_became_active":"2023-11-24T07:59:34.712482+0000","last_became_peered":"2023-11-24T07:59:34.712482+0000","last_unstale":"2023-11-24T07:59:36.317356+0000","last_undegraded":"2023-11-24T07:59:36.317356+0000","last_fullsized":"2023-11-24T07:59:36.317356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698871+0000","last_change":"2023-11-24T07:59:34.708051+0000","last_active":"2023-11-24T07:59:35.698871+0000","last_peered":"2023-11-24T07:59:35.698871+0000","last_clean":"2023-11-24T07:59:35.698871+0000","last_became_active":"2023-11-24T07:59:34.707760+0000","last_became_peered":"2023-11-24T07:59:34.707760+0000","last_unstale":"2023-11-24T07:59:35.698871+0000","last_undegraded":"2023-11-24T07:59:35.698871+0000","last_fullsized":"2023-11-24T07:59:35.698871+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:32:19.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317408+0000","last_change":"2023-11-24T07:59:34.717264+0000","last_active":"2023-11-24T07:59:36.317408+0000","last_peered":"2023-11-24T07:59:36.317408+0000","last_clean":"2023-11-24T07:59:36.317408+0000","last_became_active":"2023-11-24T07:59:34.716990+0000","last_became_peered":"2023-11-24T07:59:34.716990+0000","last_unstale":"2023-11-24T07:59:36.317408+0000","last_undegraded":"2023-11-24T07:59:36.317408+0000","last_fullsized":"2023-11-24T07:59:36.317408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702616+0000","last_change":"2023-11-24T07:59:34.701078+0000","last_active":"2023-11-24T07:59:35.702616+0000","last_peered":"2023-11-24T07:59:35.702616+0000","last_clean":"2023-11-24T07:59:35.702616+0000","last_became_active":"2023-11-24T07:59:34.700628+0000","last_became_peered":"2023-11-24T07:59:34.700628+0000","last_unstale":"2023-11-24T07:59:35.702616+0000","last_undegraded":"2023-11-24T07:59:35.702616+0000","last_fullsized":"2023-11-24T07:59:35.702616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:48.172772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317207+0000","last_change":"2023-11-24T07:59:34.707960+0000","last_active":"2023-11-24T07:59:36.317207+0000","last_peered":"2023-11-24T07:59:36.317207+0000","last_clean":"2023-11-24T07:59:36.317207+0000","last_became_active":"2023-11-24T07:59:34.707503+0000","last_became_peered":"2023-11-24T07:59:34.707503+0000","last_unstale":"2023-11-24T07:59:36.317207+0000","last_undegraded":"2023-11-24T07:59:36.317207+0000","last_fullsized":"2023-11-24T07:59:36.317207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:41.599941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316638+0000","last_change":"2023-11-24T07:59:34.706288+0000","last_active":"2023-11-24T07:59:36.316638+0000","last_peered":"2023-11-24T07:59:36.316638+0000","last_clean":"2023-11-24T07:59:36.316638+0000","last_became_active":"2023-11-24T07:59:34.706002+0000","last_became_peered":"2023-11-24T07:59:34.706002+0000","last_unstale":"2023-11-24T07:59:36.316638+0000","last_undegraded":"2023-11-24T07:59:36.316638+0000","last_fullsized":"2023-11-24T07:59:36.316638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702635+0000","last_change":"2023-11-24T07:59:34.705334+0000","last_active":"2023-11-24T07:59:35.702635+0000","last_peered":"2023-11-24T07:59:35.702635+0000","last_clean":"2023-11-24T07:59:35.702635+0000","last_became_active":"2023-11-24T07:59:34.704910+0000","last_became_peered":"2023-11-24T07:59:34.704910+0000","last_unstale":"2023-11-24T07:59:35.702635+0000","last_undegraded":"2023-11-24T07:59:35.702635+0000","last_fullsized":"2023-11-24T07:59:35.702635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:57.668606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316436+0000","last_change":"2023-11-24T07:59:34.702167+0000","last_active":"2023-11-24T07:59:36.316436+0000","last_peered":"2023-11-24T07:59:36.316436+0000","last_clean":"2023-11-24T07:59:36.316436+0000","last_became_active":"2023-11-24T07:59:34.701876+0000","last_became_peered":"2023-11-24T07:59:34.701876+0000","last_unstale":"2023-11-24T07:59:36.316436+0000","last_undegraded":"2023-11-24T07:59:36.316436+0000","last_fullsized":"2023-11-24T07:59:36.316436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:54.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316542+0000","last_change":"2023-11-24T07:59:34.705609+0000","last_active":"2023-11-24T07:59:36.316542+0000","last_peered":"2023-11-24T07:59:36.316542+0000","last_clean":"2023-11-24T07:59:36.316542+0000","last_became_active":"2023-11-24T07:59:34.704833+0000","last_became_peered":"2023-11-24T07:59:34.704833+0000","last_unstale":"2023-11-24T07:59:36.316542+0000","last_undegraded":"2023-11-24T07:59:36.316542+0000","last_fullsized":"2023-11-24T07:59:36.316542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:04.322801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702452+0000","last_change":"2023-11-24T07:59:33.696080+0000","last_active":"2023-11-24T07:59:35.702452+0000","last_peered":"2023-11-24T07:59:35.702452+0000","last_clean":"2023-11-24T07:59:35.702452+0000","last_became_active":"2023-11-24T07:59:33.695704+0000","last_became_peered":"2023-11-24T07:59:33.695704+0000","last_unstale":"2023-11-24T07:59:35.702452+0000","last_undegraded":"2023-11-24T07:59:35.702452+0000","last_fullsized":"2023-11-24T07:59:35.702452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:18.330853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702346+0000","last_change":"2023-11-24T07:59:34.707387+0000","last_active":"2023-11-24T07:59:35.702346+0000","last_peered":"2023-11-24T07:59:35.702346+0000","last_clean":"2023-11-24T07:59:35.702346+0000","last_became_active":"2023-11-24T07:59:34.705529+0000","last_became_peered":"2023-11-24T07:59:34.705529+0000","last_unstale":"2023-11-24T07:59:35.702346+0000","last_undegraded":"2023-11-24T07:59:35.702346+0000","last_fullsized":"2023-11-24T07:59:35.702346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:57.274625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316595+0000","last_change":"2023-11-24T07:59:33.702646+0000","last_active":"2023-11-24T07:59:36.316595+0000","last_peered":"2023-11-24T07:59:36.316595+0000","last_clean":"2023-11-24T07:59:36.316595+0000","last_became_active":"2023-11-24T07:59:33.702376+0000","last_became_peered":"2023-11-24T07:59:33.702376+0000","last_unstale":"2023-11-24T07:59:36.316595+0000","last_undegraded":"2023-11-24T07:59:36.316595+0000","last_fullsized":"2023-11-24T07:59:36.316595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:18.330853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317335+0000","last_change":"2023-11-24T07:59:34.710719+0000","last_active":"2023-11-24T07:59:36.317335+0000","last_peered":"2023-11-24T07:59:36.317335+0000","last_clean":"2023-11-24T07:59:36.317335+0000","last_became_active":"2023-11-24T07:59:34.710128+0000","last_became_peered":"2023-11-24T07:59:34.710128+0000","last_unstale":"2023-11-24T07:59:36.317335+0000","last_undegraded":"2023-11-24T07:59:36.317335+0000","last_fullsized":"2023-11-24T07:59:36.317335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:21.411144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698521+0000","last_change":"2023-11-24T07:59:33.699942+0000","last_active":"2023-11-24T07:59:35.698521+0000","last_peered":"2023-11-24T07:59:35.698521+0000","last_clean":"2023-11-24T07:59:35.698521+0000","last_became_active":"2023-11-24T07:59:33.699618+0000","last_became_peered":"2023-11-24T07:59:33.699618+0000","last_unstale":"2023-11-24T07:59:35.698521+0000","last_undegraded":"2023-11-24T07:59:35.698521+0000","last_fullsized":"2023-11-24T07:59:35.698521+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:56.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317405+0000","last_change":"2023-11-24T07:59:34.706643+0000","last_active":"2023-11-24T07:59:36.317405+0000","last_peered":"2023-11-24T07:59:36.317405+0000","last_clean":"2023-11-24T07:59:36.317405+0000","last_became_active":"2023-11-24T07:59:34.705694+0000","last_became_peered":"2023-11-24T07:59:34.705694+0000","last_unstale":"2023-11-24T07:59:36.317405+0000","last_undegraded":"2023-11-24T07:59:36.317405+0000","last_fullsized":"2023-11-24T07:59:36.317405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.708979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316551+0000","last_change":"2023-11-24T07:59:33.700687+0000","last_active":"2023-11-24T07:59:36.316551+0000","last_peered":"2023-11-24T07:59:36.316551+0000","last_clean":"2023-11-24T07:59:36.316551+0000","last_became_active":"2023-11-24T07:59:33.700397+0000","last_became_peered":"2023-11-24T07:59:33.700397+0000","last_unstale":"2023-11-24T07:59:36.316551+0000","last_undegraded":"2023-11-24T07:59:36.316551+0000","last_fullsized":"2023-11-24T07:59:36.316551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:40.733686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316824+0000","last_change":"2023-11-24T07:59:34.702888+0000","last_active":"2023-11-24T07:59:36.316824+0000","last_peered":"2023-11-24T07:59:36.316824+0000","last_clean":"2023-11-24T07:59:36.316824+0000","last_became_active":"2023-11-24T07:59:34.702621+0000","last_became_peered":"2023-11-24T07:59:34.702621+0000","last_unstale":"2023-11-24T07:59:36.316824+0000","last_undegraded":"2023-11-24T07:59:36.316824+0000","last_fullsized":"2023-11-24T07:59:36.316824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:59.781689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317198+0000","last_change":"2023-11-24T07:59:33.696095+0000","last_active":"2023-11-24T07:59:36.317198+0000","last_peered":"2023-11-24T07:59:36.317198+0000","last_clean":"2023-11-24T07:59:36.317198+0000","last_became_active":"2023-11-24T07:59:33.695797+0000","last_became_peered":"2023-11-24T07:59:33.695797+0000","last_unstale":"2023-11-24T07:59:36.317198+0000","last_undegraded":"2023-11-24T07:59:36.317198+0000","last_fullsized":"2023-11-24T07:59:36.317198+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:37.050060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702111+0000","last_change":"2023-11-24T07:59:34.700656+0000","last_active":"2023-11-24T07:59:35.702111+0000","last_peered":"2023-11-24T07:59:35.702111+0000","last_clean":"2023-11-24T07:59:35.702111+0000","last_became_active":"2023-11-24T07:59:34.700355+0000","last_became_peered":"2023-11-24T07:59:34.700355+0000","last_unstale":"2023-11-24T07:59:35.702111+0000","last_undegraded":"2023-11-24T07:59:35.702111+0000","last_fullsized":"2023-11-24T07:59:35.702111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:34:01.322460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317147+0000","last_change":"2023-11-24T07:59:33.696178+0000","last_active":"2023-11-24T07:59:36.317147+0000","last_peered":"2023-11-24T07:59:36.317147+0000","last_clean":"2023-11-24T07:59:36.317147+0000","last_became_active":"2023-11-24T07:59:33.695039+0000","last_became_peered":"2023-11-24T07:59:33.695039+0000","last_unstale":"2023-11-24T07:59:36.317147+0000","last_undegraded":"2023-11-24T07:59:36.317147+0000","last_fullsized":"2023-11-24T07:59:36.317147+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:22.490220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316593+0000","last_change":"2023-11-24T07:59:34.698776+0000","last_active":"2023-11-24T07:59:36.316593+0000","last_peered":"2023-11-24T07:59:36.316593+0000","last_clean":"2023-11-24T07:59:36.316593+0000","last_became_active":"2023-11-24T07:59:34.698504+0000","last_became_peered":"2023-11-24T07:59:34.698504+0000","last_unstale":"2023-11-24T07:59:36.316593+0000","last_undegraded":"2023-11-24T07:59:36.316593+0000","last_fullsized":"2023-11-24T07:59:36.316593+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:04.322801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698142+0000","last_change":"2023-11-24T07:59:33.689147+0000","last_active":"2023-11-24T07:59:35.698142+0000","last_peered":"2023-11-24T07:59:35.698142+0000","last_clean":"2023-11-24T07:59:35.698142+0000","last_became_active":"2023-11-24T07:59:33.688823+0000","last_became_peered":"2023-11-24T07:59:33.688823+0000","last_unstale":"2023-11-24T07:59:35.698142+0000","last_undegraded":"2023-11-24T07:59:35.698142+0000","last_fullsized":"2023-11-24T07:59:35.698142+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:56.663996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316861+0000","last_change":"2023-11-24T07:59:34.694778+0000","last_active":"2023-11-24T07:59:36.316861+0000","last_peered":"2023-11-24T07:59:36.316861+0000","last_clean":"2023-11-24T07:59:36.316861+0000","last_became_active":"2023-11-24T07:59:34.694462+0000","last_became_peered":"2023-11-24T07:59:34.694462+0000","last_unstale":"2023-11-24T07:59:36.316861+0000","last_undegraded":"2023-11-24T07:59:36.316861+0000","last_fullsized":"2023-11-24T07:59:36.316861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:43.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316442+0000","last_change":"2023-11-24T07:59:33.696084+0000","last_active":"2023-11-24T07:59:36.316442+0000","last_peered":"2023-11-24T07:59:36.316442+0000","last_clean":"2023-11-24T07:59:36.316442+0000","last_became_active":"2023-11-24T07:59:33.694868+0000","last_became_peered":"2023-11-24T07:59:33.694868+0000","last_unstale":"2023-11-24T07:59:36.316442+0000","last_undegraded":"2023-11-24T07:59:36.316442+0000","last_fullsized":"2023-11-24T07:59:36.316442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:03.110736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702523+0000","last_change":"2023-11-24T07:59:34.708850+0000","last_active":"2023-11-24T07:59:35.702523+0000","last_peered":"2023-11-24T07:59:35.702523+0000","last_clean":"2023-11-24T07:59:35.702523+0000","last_became_active":"2023-11-24T07:59:34.708597+0000","last_became_peered":"2023-11-24T07:59:34.708597+0000","last_unstale":"2023-11-24T07:59:35.702523+0000","last_undegraded":"2023-11-24T07:59:35.702523+0000","last_fullsized":"2023-11-24T07:59:35.702523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:18.811326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698742+0000","last_change":"2023-11-24T07:59:33.702602+0000","last_active":"2023-11-24T07:59:35.698742+0000","last_peered":"2023-11-24T07:59:35.698742+0000","last_clean":"2023-11-24T07:59:35.698742+0000","last_became_active":"2023-11-24T07:59:33.702316+0000","last_became_peered":"2023-11-24T07:59:33.702316+0000","last_unstale":"2023-11-24T07:59:35.698742+0000","last_undegraded":"2023-11-24T07:59:35.698742+0000","last_fullsized":"2023-11-24T07:59:35.698742+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:12:54.321416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698839+0000","last_change":"2023-11-24T07:59:34.717202+0000","last_active":"2023-11-24T07:59:35.698839+0000","last_peered":"2023-11-24T07:59:35.698839+0000","last_clean":"2023-11-24T07:59:35.698839+0000","last_became_active":"2023-11-24T07:59:34.716863+0000","last_became_peered":"2023-11-24T07:59:34.716863+0000","last_unstale":"2023-11-24T07:59:35.698839+0000","last_undegraded":"2023-11-24T07:59:35.698839+0000","last_fullsized":"2023-11-24T07:59:35.698839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:54.979019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316638+0000","last_change":"2023-11-24T07:59:33.694731+0000","last_active":"2023-11-24T07:59:36.316638+0000","last_peered":"2023-11-24T07:59:36.316638+0000","last_clean":"2023-11-24T07:59:36.316638+0000","last_became_active":"2023-11-24T07:59:33.694386+0000","last_became_peered":"2023-11-24T07:59:33.694386+0000","last_unstale":"2023-11-24T07:59:36.316638+0000","last_undegraded":"2023-11-24T07:59:36.316638+0000","last_fullsized":"2023-11-24T07:59:36.316638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:12:54.321416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698530+0000","last_change":"2023-11-24T07:59:34.707625+0000","last_active":"2023-11-24T07:59:35.698530+0000","last_peered":"2023-11-24T07:59:35.698530+0000","last_clean":"2023-11-24T07:59:35.698530+0000","last_became_active":"2023-11-24T07:59:34.707337+0000","last_became_peered":"2023-11-24T07:59:34.707337+0000","last_unstale":"2023-11-24T07:59:35.698530+0000","last_undegraded":"2023-11-24T07:59:35.698530+0000","last_fullsized":"2023-11-24T07:59:35.698530+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:59.781689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702076+0000","last_change":"2023-11-24T07:59:33.702355+0000","last_active":"2023-11-24T07:59:35.702076+0000","last_peered":"2023-11-24T07:59:35.702076+0000","last_clean":"2023-11-24T07:59:35.702076+0000","last_became_active":"2023-11-24T07:59:33.702011+0000","last_became_peered":"2023-11-24T07:59:33.702011+0000","last_unstale":"2023-11-24T07:59:35.702076+0000","last_undegraded":"2023-11-24T07:59:35.702076+0000","last_fullsized":"2023-11-24T07:59:35.702076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:53.702419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701989+0000","last_change":"2023-11-24T07:59:34.702344+0000","last_active":"2023-11-24T07:59:35.701989+0000","last_peered":"2023-11-24T07:59:35.701989+0000","last_clean":"2023-11-24T07:59:35.701989+0000","last_became_active":"2023-11-24T07:59:34.701869+0000","last_became_peered":"2023-11-24T07:59:34.701869+0000","last_unstale":"2023-11-24T07:59:35.701989+0000","last_undegraded":"2023-11-24T07:59:35.701989+0000","last_fullsized":"2023-11-24T07:59:35.701989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.708979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698516+0000","last_change":"2023-11-24T07:59:33.700062+0000","last_active":"2023-11-24T07:59:35.698516+0000","last_peered":"2023-11-24T07:59:35.698516+0000","last_clean":"2023-11-24T07:59:35.698516+0000","last_became_active":"2023-11-24T07:59:33.699653+0000","last_became_peered":"2023-11-24T07:59:33.699653+0000","last_unstale":"2023-11-24T07:59:35.698516+0000","last_undegraded":"2023-11-24T07:59:35.698516+0000","last_fullsized":"2023-11-24T07:59:35.698516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316904+0000","last_change":"2023-11-24T07:59:34.695614+0000","last_active":"2023-11-24T07:59:36.316904+0000","last_peered":"2023-11-24T07:59:36.316904+0000","last_clean":"2023-11-24T07:59:36.316904+0000","last_became_active":"2023-11-24T07:59:34.694400+0000","last_became_peered":"2023-11-24T07:59:34.694400+0000","last_unstale":"2023-11-24T07:59:36.316904+0000","last_undegraded":"2023-11-24T07:59:36.316904+0000","last_fullsized":"2023-11-24T07:59:36.316904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:21.250798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317301+0000","last_change":"2023-11-24T07:59:33.695933+0000","last_active":"2023-11-24T07:59:36.317301+0000","last_peered":"2023-11-24T07:59:36.317301+0000","last_clean":"2023-11-24T07:59:36.317301+0000","last_became_active":"2023-11-24T07:59:33.695629+0000","last_became_peered":"2023-11-24T07:59:33.695629+0000","last_unstale":"2023-11-24T07:59:36.317301+0000","last_undegraded":"2023-11-24T07:59:36.317301+0000","last_fullsized":"2023-11-24T07:59:36.317301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698444+0000","last_change":"2023-11-24T07:59:34.712837+0000","last_active":"2023-11-24T07:59:35.698444+0000","last_peered":"2023-11-24T07:59:35.698444+0000","last_clean":"2023-11-24T07:59:35.698444+0000","last_became_active":"2023-11-24T07:59:34.712585+0000","last_became_peered":"2023-11-24T07:59:34.712585+0000","last_unstale":"2023-11-24T07:59:35.698444+0000","last_undegraded":"2023-11-24T07:59:35.698444+0000","last_fullsized":"2023-11-24T07:59:35.698444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:31.838023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316899+0000","last_change":"2023-11-24T07:59:33.699376+0000","last_active":"2023-11-24T07:59:36.316899+0000","last_peered":"2023-11-24T07:59:36.316899+0000","last_clean":"2023-11-24T07:59:36.316899+0000","last_became_active":"2023-11-24T07:59:33.699045+0000","last_became_peered":"2023-11-24T07:59:33.699045+0000","last_unstale":"2023-11-24T07:59:36.316899+0000","last_undegraded":"2023-11-24T07:59:36.316899+0000","last_fullsized":"2023-11-24T07:59:36.316899+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316845+0000","last_change":"2023-11-24T07:59:34.702610+0000","last_active":"2023-11-24T07:59:36.316845+0000","last_peered":"2023-11-24T07:59:36.316845+0000","last_clean":"2023-11-24T07:59:36.316845+0000","last_became_active":"2023-11-24T07:59:34.702263+0000","last_became_peered":"2023-11-24T07:59:34.702263+0000","last_unstale":"2023-11-24T07:59:36.316845+0000","last_undegraded":"2023-11-24T07:59:36.316845+0000","last_fullsized":"2023-11-24T07:59:36.316845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:31.838023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698231+0000","last_change":"2023-11-24T07:59:33.702517+0000","last_active":"2023-11-24T07:59:35.698231+0000","last_peered":"2023-11-24T07:59:35.698231+0000","last_clean":"2023-11-24T07:59:35.698231+0000","last_became_active":"2023-11-24T07:59:33.702216+0000","last_became_peered":"2023-11-24T07:59:33.702216+0000","last_unstale":"2023-11-24T07:59:35.698231+0000","last_undegraded":"2023-11-24T07:59:35.698231+0000","last_fullsized":"2023-11-24T07:59:35.698231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:57.344256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701952+0000","last_change":"2023-11-24T07:59:34.704646+0000","last_active":"2023-11-24T07:59:35.701952+0000","last_peered":"2023-11-24T07:59:35.701952+0000","last_clean":"2023-11-24T07:59:35.701952+0000","last_became_active":"2023-11-24T07:59:34.704140+0000","last_became_peered":"2023-11-24T07:59:34.704140+0000","last_unstale":"2023-11-24T07:59:35.701952+0000","last_undegraded":"2023-11-24T07:59:35.701952+0000","last_fullsized":"2023-11-24T07:59:35.701952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:38.054671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698038+0000","last_change":"2023-11-24T07:59:33.697712+0000","last_active":"2023-11-24T07:59:35.698038+0000","last_peered":"2023-11-24T07:59:35.698038+0000","last_clean":"2023-11-24T07:59:35.698038+0000","last_became_active":"2023-11-24T07:59:33.697416+0000","last_became_peered":"2023-11-24T07:59:33.697416+0000","last_unstale":"2023-11-24T07:59:35.698038+0000","last_undegraded":"2023-11-24T07:59:35.698038+0000","last_fullsized":"2023-11-24T07:59:35.698038+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:22.490220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317458+0000","last_change":"2023-11-24T07:59:34.698174+0000","last_active":"2023-11-24T07:59:36.317458+0000","last_peered":"2023-11-24T07:59:36.317458+0000","last_clean":"2023-11-24T07:59:36.317458+0000","last_became_active":"2023-11-24T07:59:34.697893+0000","last_became_peered":"2023-11-24T07:59:34.697893+0000","last_unstale":"2023-11-24T07:59:36.317458+0000","last_undegraded":"2023-11-24T07:59:36.317458+0000","last_fullsized":"2023-11-24T07:59:36.317458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:28.294849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317469+0000","last_change":"2023-11-24T07:59:33.697751+0000","last_active":"2023-11-24T07:59:36.317469+0000","last_peered":"2023-11-24T07:59:36.317469+0000","last_clean":"2023-11-24T07:59:36.317469+0000","last_became_active":"2023-11-24T07:59:33.696926+0000","last_became_peered":"2023-11-24T07:59:33.696926+0000","last_unstale":"2023-11-24T07:59:36.317469+0000","last_undegraded":"2023-11-24T07:59:36.317469+0000","last_fullsized":"2023-11-24T07:59:36.317469+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:03.318191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702361+0000","last_change":"2023-11-24T07:59:34.700908+0000","last_active":"2023-11-24T07:59:35.702361+0000","last_peered":"2023-11-24T07:59:35.702361+0000","last_clean":"2023-11-24T07:59:35.702361+0000","last_became_active":"2023-11-24T07:59:34.700430+0000","last_became_peered":"2023-11-24T07:59:34.700430+0000","last_unstale":"2023-11-24T07:59:35.702361+0000","last_undegraded":"2023-11-24T07:59:35.702361+0000","last_fullsized":"2023-11-24T07:59:35.702361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:59.781689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317511+0000","last_change":"2023-11-24T07:59:33.695522+0000","last_active":"2023-11-24T07:59:36.317511+0000","last_peered":"2023-11-24T07:59:36.317511+0000","last_clean":"2023-11-24T07:59:36.317511+0000","last_became_active":"2023-11-24T07:59:33.694756+0000","last_became_peered":"2023-11-24T07:59:33.694756+0000","last_unstale":"2023-11-24T07:59:36.317511+0000","last_undegraded":"2023-11-24T07:59:36.317511+0000","last_fullsized":"2023-11-24T07:59:36.317511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:56.663996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317243+0000","last_change":"2023-11-24T07:59:34.707523+0000","last_active":"2023-11-24T07:59:36.317243+0000","last_peered":"2023-11-24T07:59:36.317243+0000","last_clean":"2023-11-24T07:59:36.317243+0000","last_became_active":"2023-11-24T07:59:34.706844+0000","last_became_peered":"2023-11-24T07:59:34.706844+0000","last_unstale":"2023-11-24T07:59:36.317243+0000","last_undegraded":"2023-11-24T07:59:36.317243+0000","last_fullsized":"2023-11-24T07:59:36.317243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:54.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317196+0000","last_change":"2023-11-24T07:59:33.694599+0000","last_active":"2023-11-24T07:59:36.317196+0000","last_peered":"2023-11-24T07:59:36.317196+0000","last_clean":"2023-11-24T07:59:36.317196+0000","last_became_active":"2023-11-24T07:59:33.693972+0000","last_became_peered":"2023-11-24T07:59:33.693972+0000","last_unstale":"2023-11-24T07:59:36.317196+0000","last_undegraded":"2023-11-24T07:59:36.317196+0000","last_fullsized":"2023-11-24T07:59:36.317196+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:39.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316513+0000","last_change":"2023-11-24T07:59:34.708238+0000","last_active":"2023-11-24T07:59:36.316513+0000","last_peered":"2023-11-24T07:59:36.316513+0000","last_clean":"2023-11-24T07:59:36.316513+0000","last_became_active":"2023-11-24T07:59:34.707944+0000","last_became_peered":"2023-11-24T07:59:34.707944+0000","last_unstale":"2023-11-24T07:59:36.316513+0000","last_undegraded":"2023-11-24T07:59:36.316513+0000","last_fullsized":"2023-11-24T07:59:36.316513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:11.165477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317502+0000","last_change":"2023-11-24T07:59:33.696954+0000","last_active":"2023-11-24T07:59:36.317502+0000","last_peered":"2023-11-24T07:59:36.317502+0000","last_clean":"2023-11-24T07:59:36.317502+0000","last_became_active":"2023-11-24T07:59:33.696594+0000","last_became_peered":"2023-11-24T07:59:33.696594+0000","last_unstale":"2023-11-24T07:59:36.317502+0000","last_undegraded":"2023-11-24T07:59:36.317502+0000","last_fullsized":"2023-11-24T07:59:36.317502+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:12:54.321416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316210+0000","last_change":"2023-11-24T07:59:34.701913+0000","last_active":"2023-11-24T07:59:36.316210+0000","last_peered":"2023-11-24T07:59:36.316210+0000","last_clean":"2023-11-24T07:59:36.316210+0000","last_became_active":"2023-11-24T07:59:34.701254+0000","last_became_peered":"2023-11-24T07:59:34.701254+0000","last_unstale":"2023-11-24T07:59:36.316210+0000","last_undegraded":"2023-11-24T07:59:36.316210+0000","last_fullsized":"2023-11-24T07:59:36.316210+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:18:56.985255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316226+0000","last_change":"2023-11-24T07:59:33.700636+0000","last_active":"2023-11-24T07:59:36.316226+0000","last_peered":"2023-11-24T07:59:36.316226+0000","last_clean":"2023-11-24T07:59:36.316226+0000","last_became_active":"2023-11-24T07:59:33.700236+0000","last_became_peered":"2023-11-24T07:59:33.700236+0000","last_unstale":"2023-11-24T07:59:36.316226+0000","last_undegraded":"2023-11-24T07:59:36.316226+0000","last_fullsized":"2023-11-24T07:59:36.316226+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:02.256736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316652+0000","last_change":"2023-11-24T07:59:34.713518+0000","last_active":"2023-11-24T07:59:36.316652+0000","last_peered":"2023-11-24T07:59:36.316652+0000","last_clean":"2023-11-24T07:59:36.316652+0000","last_became_active":"2023-11-24T07:59:34.713162+0000","last_became_peered":"2023-11-24T07:59:34.713162+0000","last_unstale":"2023-11-24T07:59:36.316652+0000","last_undegraded":"2023-11-24T07:59:36.316652+0000","last_fullsized":"2023-11-24T07:59:36.316652+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:40.480402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702504+0000","last_change":"2023-11-24T07:59:33.700732+0000","last_active":"2023-11-24T07:59:35.702504+0000","last_peered":"2023-11-24T07:59:35.702504+0000","last_clean":"2023-11-24T07:59:35.702504+0000","last_became_active":"2023-11-24T07:59:33.700454+0000","last_became_peered":"2023-11-24T07:59:33.700454+0000","last_unstale":"2023-11-24T07:59:35.702504+0000","last_undegraded":"2023-11-24T07:59:35.702504+0000","last_fullsized":"2023-11-24T07:59:35.702504+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:46:39.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316163+0000","last_change":"2023-11-24T07:59:34.709505+0000","last_active":"2023-11-24T07:59:36.316163+0000","last_peered":"2023-11-24T07:59:36.316163+0000","last_clean":"2023-11-24T07:59:36.316163+0000","last_became_active":"2023-11-24T07:59:34.706917+0000","last_became_peered":"2023-11-24T07:59:34.706917+0000","last_unstale":"2023-11-24T07:59:36.316163+0000","last_undegraded":"2023-11-24T07:59:36.316163+0000","last_fullsized":"2023-11-24T07:59:36.316163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:58:50.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316917+0000","last_change":"2023-11-24T07:59:33.700333+0000","last_active":"2023-11-24T07:59:36.316917+0000","last_peered":"2023-11-24T07:59:36.316917+0000","last_clean":"2023-11-24T07:59:36.316917+0000","last_became_active":"2023-11-24T07:59:33.699990+0000","last_became_peered":"2023-11-24T07:59:33.699990+0000","last_unstale":"2023-11-24T07:59:36.316917+0000","last_undegraded":"2023-11-24T07:59:36.316917+0000","last_fullsized":"2023-11-24T07:59:36.316917+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:31:10.160866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316889+0000","last_change":"2023-11-24T07:59:34.706961+0000","last_active":"2023-11-24T07:59:36.316889+0000","last_peered":"2023-11-24T07:59:36.316889+0000","last_clean":"2023-11-24T07:59:36.316889+0000","last_became_active":"2023-11-24T07:59:34.706269+0000","last_became_peered":"2023-11-24T07:59:34.706269+0000","last_unstale":"2023-11-24T07:59:36.316889+0000","last_undegraded":"2023-11-24T07:59:36.316889+0000","last_fullsized":"2023-11-24T07:59:36.316889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:18.811326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.701831+0000","last_change":"2023-11-24T07:59:33.703879+0000","last_active":"2023-11-24T07:59:35.701831+0000","last_peered":"2023-11-24T07:59:35.701831+0000","last_clean":"2023-11-24T07:59:35.701831+0000","last_became_active":"2023-11-24T07:59:33.703565+0000","last_became_peered":"2023-11-24T07:59:33.703565+0000","last_unstale":"2023-11-24T07:59:35.701831+0000","last_undegraded":"2023-11-24T07:59:35.701831+0000","last_fullsized":"2023-11-24T07:59:35.701831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:40.595331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316678+0000","last_change":"2023-11-24T07:59:34.698078+0000","last_active":"2023-11-24T07:59:36.316678+0000","last_peered":"2023-11-24T07:59:36.316678+0000","last_clean":"2023-11-24T07:59:36.316678+0000","last_became_active":"2023-11-24T07:59:34.697740+0000","last_became_peered":"2023-11-24T07:59:34.697740+0000","last_unstale":"2023-11-24T07:59:36.316678+0000","last_undegraded":"2023-11-24T07:59:36.316678+0000","last_fullsized":"2023-11-24T07:59:36.316678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:04.115346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317347+0000","last_change":"2023-11-24T07:59:33.697566+0000","last_active":"2023-11-24T07:59:36.317347+0000","last_peered":"2023-11-24T07:59:36.317347+0000","last_clean":"2023-11-24T07:59:36.317347+0000","last_became_active":"2023-11-24T07:59:33.697280+0000","last_became_peered":"2023-11-24T07:59:33.697280+0000","last_unstale":"2023-11-24T07:59:36.317347+0000","last_undegraded":"2023-11-24T07:59:36.317347+0000","last_fullsized":"2023-11-24T07:59:36.317347+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:22.490220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702209+0000","last_change":"2023-11-24T07:59:34.716265+0000","last_active":"2023-11-24T07:59:35.702209+0000","last_peered":"2023-11-24T07:59:35.702209+0000","last_clean":"2023-11-24T07:59:35.702209+0000","last_became_active":"2023-11-24T07:59:34.715970+0000","last_became_peered":"2023-11-24T07:59:34.715970+0000","last_unstale":"2023-11-24T07:59:35.702209+0000","last_undegraded":"2023-11-24T07:59:35.702209+0000","last_fullsized":"2023-11-24T07:59:35.702209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:10.747185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316677+0000","last_change":"2023-11-24T07:59:33.697890+0000","last_active":"2023-11-24T07:59:36.316677+0000","last_peered":"2023-11-24T07:59:36.316677+0000","last_clean":"2023-11-24T07:59:36.316677+0000","last_became_active":"2023-11-24T07:59:33.697587+0000","last_became_peered":"2023-11-24T07:59:33.697587+0000","last_unstale":"2023-11-24T07:59:36.316677+0000","last_undegraded":"2023-11-24T07:59:36.316677+0000","last_fullsized":"2023-11-24T07:59:36.316677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:37.050060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316061+0000","last_change":"2023-11-24T07:59:34.717080+0000","last_active":"2023-11-24T07:59:36.316061+0000","last_peered":"2023-11-24T07:59:36.316061+0000","last_clean":"2023-11-24T07:59:36.316061+0000","last_became_active":"2023-11-24T07:59:34.716798+0000","last_became_peered":"2023-11-24T07:59:34.716798+0000","last_unstale":"2023-11-24T07:59:36.316061+0000","last_undegraded":"2023-11-24T07:59:36.316061+0000","last_fullsized":"2023-11-24T07:59:36.316061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:58.348867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317302+0000","last_change":"2023-11-24T07:59:33.700926+0000","last_active":"2023-11-24T07:59:36.317302+0000","last_peered":"2023-11-24T07:59:36.317302+0000","last_clean":"2023-11-24T07:59:36.317302+0000","last_became_active":"2023-11-24T07:59:33.700542+0000","last_became_peered":"2023-11-24T07:59:33.700542+0000","last_unstale":"2023-11-24T07:59:36.317302+0000","last_undegraded":"2023-11-24T07:59:36.317302+0000","last_fullsized":"2023-11-24T07:59:36.317302+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:27:57.344256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702418+0000","last_change":"2023-11-24T07:59:34.709122+0000","last_active":"2023-11-24T07:59:35.702418+0000","last_peered":"2023-11-24T07:59:35.702418+0000","last_clean":"2023-11-24T07:59:35.702418+0000","last_became_active":"2023-11-24T07:59:34.708781+0000","last_became_peered":"2023-11-24T07:59:34.708781+0000","last_unstale":"2023-11-24T07:59:35.702418+0000","last_undegraded":"2023-11-24T07:59:35.702418+0000","last_fullsized":"2023-11-24T07:59:35.702418+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:03.261346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316191+0000","last_change":"2023-11-24T07:59:33.696176+0000","last_active":"2023-11-24T07:59:36.316191+0000","last_peered":"2023-11-24T07:59:36.316191+0000","last_clean":"2023-11-24T07:59:36.316191+0000","last_became_active":"2023-11-24T07:59:33.695894+0000","last_became_peered":"2023-11-24T07:59:33.695894+0000","last_unstale":"2023-11-24T07:59:36.316191+0000","last_undegraded":"2023-11-24T07:59:36.316191+0000","last_fullsized":"2023-11-24T07:59:36.316191+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:29:56.270014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316326+0000","last_change":"2023-11-24T07:59:34.712932+0000","last_active":"2023-11-24T07:59:36.316326+0000","last_peered":"2023-11-24T07:59:36.316326+0000","last_clean":"2023-11-24T07:59:36.316326+0000","last_became_active":"2023-11-24T07:59:34.712521+0000","last_became_peered":"2023-11-24T07:59:34.712521+0000","last_unstale":"2023-11-24T07:59:36.316326+0000","last_undegraded":"2023-11-24T07:59:36.316326+0000","last_fullsized":"2023-11-24T07:59:36.316326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:23.494831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317007+0000","last_change":"2023-11-24T07:59:33.699949+0000","last_active":"2023-11-24T07:59:36.317007+0000","last_peered":"2023-11-24T07:59:36.317007+0000","last_clean":"2023-11-24T07:59:36.317007+0000","last_became_active":"2023-11-24T07:59:33.699614+0000","last_became_peered":"2023-11-24T07:59:33.699614+0000","last_unstale":"2023-11-24T07:59:36.317007+0000","last_undegraded":"2023-11-24T07:59:36.317007+0000","last_fullsized":"2023-11-24T07:59:36.317007+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:56.663996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698078+0000","last_change":"2023-11-24T07:59:34.691124+0000","last_active":"2023-11-24T07:59:35.698078+0000","last_peered":"2023-11-24T07:59:35.698078+0000","last_clean":"2023-11-24T07:59:35.698078+0000","last_became_active":"2023-11-24T07:59:34.690758+0000","last_became_peered":"2023-11-24T07:59:34.690758+0000","last_unstale":"2023-11-24T07:59:35.698078+0000","last_undegraded":"2023-11-24T07:59:35.698078+0000","last_fullsized":"2023-11-24T07:59:35.698078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:10.747185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.317026+0000","last_change":"2023-11-24T07:59:33.696331+0000","last_active":"2023-11-24T07:59:36.317026+0000","last_peered":"2023-11-24T07:59:36.317026+0000","last_clean":"2023-11-24T07:59:36.317026+0000","last_became_active":"2023-11-24T07:59:33.694494+0000","last_became_peered":"2023-11-24T07:59:33.694494+0000","last_unstale":"2023-11-24T07:59:36.317026+0000","last_undegraded":"2023-11-24T07:59:36.317026+0000","last_fullsized":"2023-11-24T07:59:36.317026+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:05.919483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316744+0000","last_change":"2023-11-24T07:59:34.707779+0000","last_active":"2023-11-24T07:59:36.316744+0000","last_peered":"2023-11-24T07:59:36.316744+0000","last_clean":"2023-11-24T07:59:36.316744+0000","last_became_active":"2023-11-24T07:59:34.707540+0000","last_became_peered":"2023-11-24T07:59:34.707540+0000","last_unstale":"2023-11-24T07:59:36.316744+0000","last_undegraded":"2023-11-24T07:59:36.316744+0000","last_fullsized":"2023-11-24T07:59:36.316744+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:43.141545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.698166+0000","last_change":"2023-11-24T07:59:33.703107+0000","last_active":"2023-11-24T07:59:35.698166+0000","last_peered":"2023-11-24T07:59:35.698166+0000","last_clean":"2023-11-24T07:59:35.698166+0000","last_became_active":"2023-11-24T07:59:33.702844+0000","last_became_peered":"2023-11-24T07:59:33.702844+0000","last_unstale":"2023-11-24T07:59:35.698166+0000","last_undegraded":"2023-11-24T07:59:35.698166+0000","last_fullsized":"2023-11-24T07:59:35.698166+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:42.136935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:36.316637+0000","last_change":"2023-11-24T07:59:34.708726+0000","last_active":"2023-11-24T07:59:36.316637+0000","last_peered":"2023-11-24T07:59:36.316637+0000","last_clean":"2023-11-24T07:59:36.316637+0000","last_became_active":"2023-11-24T07:59:34.708433+0000","last_became_peered":"2023-11-24T07:59:34.708433+0000","last_unstale":"2023-11-24T07:59:36.316637+0000","last_undegraded":"2023-11-24T07:59:36.316637+0000","last_fullsized":"2023-11-24T07:59:36.316637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:33.674155+0000","last_clean_scrub_stamp":"2023-11-24T07:59:33.674155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:22.708979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:59:35.702200+0000","last_change":"2023-11-24T07:59:33.695863+0000","last_active":"2023-11-24T07:59:35.702200+0000","last_peered":"2023-11-24T07:59:35.702200+0000","last_clean":"2023-11-24T07:59:35.702200+0000","last_became_active":"2023-11-24T07:59:33.695557+0000","last_became_peered":"2023-11-24T07:59:33.695557+0000","last_unstale":"2023-11-24T07:59:35.702200+0000","last_undegraded":"2023-11-24T07:59:35.702200+0000","last_fullsized":"2023-11-24T07:59:35.702200+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:32.669545+0000","last_clean_scrub_stamp":"2023-11-24T07:59:32.669545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:12.814684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-24T07:59:34.688534+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:59:34.688534+0000","last_peered":"2023-11-24T07:59:34.688534+0000","last_clean":"2023-11-24T07:59:34.688534+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:59:34.688534+0000","last_undegraded":"2023-11-24T07:59:34.688534+0000","last_fullsized":"2023-11-24T07:59:34.688534+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":729231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.62}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.70899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":139389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.71299999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.623},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":140407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.751}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.316}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.37},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.433}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.49299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":132543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.377}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.62},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":134107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.751},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.78800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":138877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.621},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.55000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":139445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.748}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:39.252 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:59:39.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T07:59:39.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.414+0000 7fa10270b700 1 Processor -- start 2023-11-24T07:59:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.414+0000 7fa10270b700 1 -- start start 2023-11-24T07:59:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa10270b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0fc06f440 0x7fa0fc0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa10270b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0fc073f90 0x7fa0fc074440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa10270b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc0cd830 con 0x7fa0fc15c000 2023-11-24T07:59:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa10270b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc074980 con 0x7fa0fc06f440 2023-11-24T07:59:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa10270b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc06dc70 con 0x7fa0fc073f90 2023-11-24T07:59:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0fc06f440 0x7fa0fc0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0fc06f440 0x7fa0fc0d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58456/0 (socket says 172.21.15.32:58456) 2023-11-24T07:59:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0fc15c000 0x7fa0fc06e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60166/0 (socket says 172.21.15.32:60166) 2023-11-24T07:59:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 -- 172.21.15.32:0/385266203 learned_addr learned my addr 172.21.15.32:0/385266203 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa100ca8700 1 --2- 172.21.15.32:0/385266203 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc073f90 0x7fa0fc074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 -- 172.21.15.32:0/385266203 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc073f90 msgr2=0x7fa0fc074440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 --2- 172.21.15.32:0/385266203 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc073f90 0x7fa0fc074440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 -- 172.21.15.32:0/385266203 >> v1:172.21.15.32:6789/0 conn(0x7fa0fc15c000 legacy=0x7fa0fc06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 -- 172.21.15.32:0/385266203 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc06eb40 con 0x7fa0fc06f440 2023-11-24T07:59:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa100ca8700 1 --2- 172.21.15.32:0/385266203 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc073f90 0x7fa0fc074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0fb7fe700 1 --2- 172.21.15.32:0/385266203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc0d06a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc072f20 tx=0x7fa0e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d2dac620e8f268 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.415+0000 7fa0faffd700 1 -- 172.21.15.32:0/385266203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400b870 con 0x7fa0fc06f440 2023-11-24T07:59:39.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa0faffd700 1 -- 172.21.15.32:0/385266203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400b9d0 con 0x7fa0fc06f440 2023-11-24T07:59:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa0faffd700 1 -- 172.21.15.32:0/385266203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400bd60 con 0x7fa0fc06f440 2023-11-24T07:59:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 -- 172.21.15.32:0/385266203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 msgr2=0x7fa0fc0d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 --2- 172.21.15.32:0/385266203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc0d06a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc072f20 tx=0x7fa0e4009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 -- 172.21.15.32:0/385266203 shutdown_connections 2023-11-24T07:59:39.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 --2- 172.21.15.32:0/385266203 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc073f90 0x7fa0fc074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 --2- 172.21.15.32:0/385266203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc0d06a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 -- 172.21.15.32:0/385266203 >> 172.21.15.32:0/385266203 conn(0x7fa0fc072960 msgr2=0x7fa0fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 -- 172.21.15.32:0/385266203 shutdown_connections 2023-11-24T07:59:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.416+0000 7fa10270b700 1 -- 172.21.15.32:0/385266203 wait complete. 2023-11-24T07:59:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.417+0000 7fa10270b700 1 Processor -- start 2023-11-24T07:59:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.417+0000 7fa10270b700 1 -- start start 2023-11-24T07:59:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa10270b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa10270b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc073f90 0x7fa0fc14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa10270b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc15c000 0x7fa0fc0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa10270b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc159180 con 0x7fa0fc06f440 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa10270b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc159460 con 0x7fa0fc15c000 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa10270b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc159740 con 0x7fa0fc073f90 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc15a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58460/0 (socket says 172.21.15.32:58460) 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.418+0000 7fa0fbfff700 1 -- 172.21.15.32:0/4112517403 learned_addr learned my addr 172.21.15.32:0/4112517403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0fbfff700 1 -- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc073f90 msgr2=0x7fa0fc14b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0fbfff700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc073f90 0x7fa0fc14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0fbfff700 1 -- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc15c000 msgr2=0x7fa0fc0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa100ca8700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc15c000 0x7fa0fc0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0fbfff700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc15c000 0x7fa0fc0b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0fbfff700 1 -- 172.21.15.32:0/4112517403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4009550 con 0x7fa0fc06f440 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0fbfff700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc15a8a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c500 tx=0x7fa0ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec005cb0 con 0x7fa0fc06f440 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec005e10 con 0x7fa0fc06f440 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc159a20 con 0x7fa0fc06f440 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0072d0 con 0x7fa0fc06f440 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.419+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc1f3da0 con 0x7fa0fc06f440 2023-11-24T07:59:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.420+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc0af250 con 0x7fa0fc06f440 2023-11-24T07:59:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.420+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0ec007430 con 0x7fa0fc06f440 2023-11-24T07:59:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.421+0000 7fa0f8ff9700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0dc06aed0 0x7fa0dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.421+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec088910 con 0x7fa0fc06f440 2023-11-24T07:59:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.421+0000 7fa0fb7fe700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0dc06aed0 0x7fa0dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.422+0000 7fa0fb7fe700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0dc06aed0 0x7fa0dc06d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc072f20 tx=0x7fa0e4010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.425+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0ec053a70 con 0x7fa0fc06f440 2023-11-24T07:59:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.712+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa0fc0af0c0 con 0x7fa0fc06f440 2023-11-24T07:59:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.714+0000 7fa0f8ff9700 1 -- 172.21.15.32:0/4112517403 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa0ec017020 con 0x7fa0fc06f440 2023-11-24T07:59:39.716 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-11-24T07:59:39.717 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQB4V2Bl4D6UBxAADXMYHy5S2dPfRBlHDFLvHg== 2023-11-24T07:59:39.717 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2023-11-24T07:59:39.717 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2023-11-24T07:59:39.717 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T07:59:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0dc06aed0 msgr2=0x7fa0dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0dc06aed0 0x7fa0dc06d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc072f20 tx=0x7fa0e4010000 comp rx=0 tx=0).stop 2023-11-24T07:59:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 msgr2=0x7fa0fc15a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc15a8a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c500 tx=0x7fa0ec00c8c0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 shutdown_connections 2023-11-24T07:59:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0dc06aed0 0x7fa0dc06d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc15c000 0x7fa0fc0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0fc06f440 0x7fa0fc15a8a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 --2- 172.21.15.32:0/4112517403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc073f90 0x7fa0fc14b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.717+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 >> 172.21.15.32:0/4112517403 conn(0x7fa0fc072960 msgr2=0x7fa0fc15fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.718+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 shutdown_connections 2023-11-24T07:59:39.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.718+0000 7fa10270b700 1 -- 172.21.15.32:0/4112517403 wait complete. 2023-11-24T07:59:39.723 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-11-24T07:59:39.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:39.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.857+0000 7fbbe49d3700 1 Processor -- start 2023-11-24T07:59:39.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.857+0000 7fbbe49d3700 1 -- start start 2023-11-24T07:59:39.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.858+0000 7fbbe49d3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbe015e080 0x7fbbe00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.858+0000 7fbbe49d3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00ae430 con 0x7fbbe00af4d0 2023-11-24T07:59:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.858+0000 7fbbe49d3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00ae710 con 0x7fbbe006c710 2023-11-24T07:59:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.858+0000 7fbbe49d3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe00af0b0 con 0x7fbbe015e080 2023-11-24T07:59:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbddd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbbe00af4d0 0x7fbbe006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60180/0 (socket says 172.21.15.32:60180) 2023-11-24T07:59:39.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbddd9b700 1 -- 172.21.15.32:0/888641117 learned_addr learned my addr 172.21.15.32:0/888641117 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbde59c700 1 --2- 172.21.15.32:0/888641117 >> v2:172.21.15.123:3300/0 conn(0x7fbbe015e080 0x7fbbe00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123697087 0 0) 0x7fbbe00ae430 con 0x7fbbe00af4d0 2023-11-24T07:59:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbcc0035b0 con 0x7fbbe00af4d0 2023-11-24T07:59:39.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908317048 0 0) 0x7fbbcc0035b0 con 0x7fbbe00af4d0 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 >> v2:172.21.15.123:3300/0 conn(0x7fbbe015e080 msgr2=0x7fbbe00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 --2- 172.21.15.32:0/888641117 >> v2:172.21.15.123:3300/0 conn(0x7fbbe015e080 0x7fbbe00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 >> v1:172.21.15.123:6789/0 conn(0x7fbbe006c710 legacy=0x7fbbe006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00ae9f0 con 0x7fbbe00af4d0 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fbbc8001670 con 0x7fbbe00af4d0 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 >> v1:172.21.15.32:6789/0 conn(0x7fbbe00af4d0 legacy=0x7fbbe006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbcc003a00 0x7fbbcc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbcc004850 0x7fbbcc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc005210 0x7fbbcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00af0b0 con 0x7fbbcc003a00 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc00c0b0 con 0x7fbbcc005210 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc00c270 con 0x7fbbcc004850 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbde59c700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbcc003a00 0x7fbbcc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbddd9b700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbcc004850 0x7fbbcc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.859+0000 7fbbded9d700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc005210 0x7fbbcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbde59c700 1 -- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbcc004850 msgr2=0x7fbbcc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbde59c700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbcc004850 0x7fbbcc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbde59c700 1 -- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc005210 msgr2=0x7fbbcc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbde59c700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc005210 0x7fbbcc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbde59c700 1 -- 172.21.15.32:0/888641117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0035b0 con 0x7fbbcc003a00 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbded9d700 1 -- 172.21.15.32:0/888641117 reap_dead start 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbde59c700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbcc003a00 0x7fbbcc003e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009b60 tx=0x7fbbcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5c3e111c6e49df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000e6c0 con 0x7fbbcc003a00 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd000e820 con 0x7fbbcc003a00 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbdd59a700 1 -- 172.21.15.32:0/888641117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000eb90 con 0x7fbbcc003a00 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbe49d3700 1 -- 172.21.15.32:0/888641117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbcc003a00 msgr2=0x7fbbcc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.860+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbcc003a00 0x7fbbcc003e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009b60 tx=0x7fbbcc005880 comp rx=0 tx=0).stop 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.861+0000 7fbbe49d3700 1 -- 172.21.15.32:0/888641117 shutdown_connections 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.861+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/888641117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbcc003a00 0x7fbbcc003e20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.861+0000 7fbbe49d3700 1 -- 172.21.15.32:0/888641117 >> 172.21.15.32:0/888641117 conn(0x7fbbe0071b90 msgr2=0x7fbbe0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.861+0000 7fbbe49d3700 1 -- 172.21.15.32:0/888641117 shutdown_connections 2023-11-24T07:59:39.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.861+0000 7fbbe49d3700 1 -- 172.21.15.32:0/888641117 wait complete. 2023-11-24T07:59:39.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.862+0000 7fbbe49d3700 1 Processor -- start 2023-11-24T07:59:39.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.862+0000 7fbbe49d3700 1 -- start start 2023-11-24T07:59:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbe49d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc003a00 0x7fbbe0143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbe49d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 0x7fbbe01f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbe49d3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe01f73c0 0x7fbbe00b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbe49d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0155b30 con 0x7fbbe0144280 2023-11-24T07:59:39.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbe49d3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01f7900 con 0x7fbbcc003a00 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbe49d3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01f7be0 con 0x7fbbe01f73c0 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 0x7fbbe01f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 0x7fbbe01f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58482/0 (socket says 172.21.15.32:58482) 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 -- 172.21.15.32:0/1511119893 learned_addr learned my addr 172.21.15.32:0/1511119893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbded9d700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe01f73c0 0x7fbbe00b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbde59c700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc003a00 0x7fbbe0143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 -- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe01f73c0 msgr2=0x7fbbe00b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe01f73c0 0x7fbbe00b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 -- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc003a00 msgr2=0x7fbbe0143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc003a00 0x7fbbe0143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 -- 172.21.15.32:0/1511119893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe01557c0 con 0x7fbbe0144280 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbde59c700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc003a00 0x7fbbe0143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbddd9b700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 0x7fbbe01f6e80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80011e0 tx=0x7fbbc8007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800f5e0 con 0x7fbbe0144280 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.863+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc800f760 con 0x7fbbe0144280 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.864+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe0155470 con 0x7fbbe0144280 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.864+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc800fb10 con 0x7fbbe0144280 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.864+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00b1830 con 0x7fbbe0144280 2023-11-24T07:59:39.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.865+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe0155e10 con 0x7fbbe0144280 2023-11-24T07:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.865+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbbc800fc70 con 0x7fbbe0144280 2023-11-24T07:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.866+0000 7fbbd6ffd700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbbc006aed0 0x7fbbc006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.866+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbc8089590 con 0x7fbbe0144280 2023-11-24T07:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.866+0000 7fbbde59c700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbbc006aed0 0x7fbbc006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.866+0000 7fbbde59c700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbbc006aed0 0x7fbbc006d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00090b0 tx=0x7fbbd0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:39.870+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbbc8054650 con 0x7fbbe0144280 2023-11-24T07:59:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.115+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbe00753f0 con 0x7fbbe0144280 2023-11-24T07:59:40.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.115+0000 7fbbd6ffd700 1 -- 172.21.15.32:0/1511119893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fbbc8057c70 con 0x7fbbe0144280 2023-11-24T07:59:40.117 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:40.117 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:34.106005+0000","modified":"2023-11-24T07:59:37.114690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/598023464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":598023464},{"type":"v1","addr":"172.21.15.32:6835","nonce":598023464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:59:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbbc006aed0 msgr2=0x7fbbc006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbbc006aed0 0x7fbbc006d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00090b0 tx=0x7fbbd0008030 comp rx=0 tx=0).stop 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 msgr2=0x7fbbe01f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 0x7fbbe01f6e80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80011e0 tx=0x7fbbc8007570 comp rx=0 tx=0).stop 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 shutdown_connections 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbbc006aed0 0x7fbbc006d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbe01f73c0 0x7fbbe00b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbcc003a00 0x7fbbe0143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 --2- 172.21.15.32:0/1511119893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe0144280 0x7fbbe01f6e80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 >> 172.21.15.32:0/1511119893 conn(0x7fbbe0071b90 msgr2=0x7fbbe006e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 shutdown_connections 2023-11-24T07:59:40.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.118+0000 7fbbe49d3700 1 -- 172.21.15.32:0/1511119893 wait complete. 2023-11-24T07:59:40.121 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-11-24T07:59:40.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:59:34.106005+0000', 'modified': '2023-11-24T07:59:37.114690+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5106}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5106': {'gid': 5106, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6835/598023464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6833', 'nonce': 598023464}, {'type': 'v1', 'addr': '172.21.15.32:6835', 'nonce': 598023464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:59:40.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:59:40.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T07:59:40.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.663+0000 7f646ecff700 1 Processor -- start 2023-11-24T07:59:40.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.663+0000 7f646ecff700 1 -- start start 2023-11-24T07:59:40.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f646ecff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 0x7f6468165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f646ecff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64680b4290 con 0x7f646815f180 2023-11-24T07:59:40.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f646ecff700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64680d0380 con 0x7f646813e000 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f646ecff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646813e570 con 0x7f64680cff10 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f6467fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f646815f180 0x7f64680cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60200/0 (socket says 172.21.15.32:60200) 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f6467fff700 1 -- 172.21.15.32:0/1815774007 learned_addr learned my addr 172.21.15.32:0/1815774007 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321726940 0 0) 0x7f64680b4290 con 0x7f646815f180 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f646d29c700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 0x7f6468165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64580035b0 con 0x7f646815f180 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853359100 0 0) 0x7f64580035b0 con 0x7f646815f180 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 msgr2=0x7f6468165bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 0x7f6468165bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 >> v1:172.21.15.123:6789/0 conn(0x7f646813e000 legacy=0x7f646815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680d05d0 con 0x7f646815f180 2023-11-24T07:59:40.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.664+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f6454001670 con 0x7f646815f180 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 >> v1:172.21.15.32:6789/0 conn(0x7f646815f180 legacy=0x7f64680cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 0x7f6458003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6458003fb0 0x7f6458004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6458004d90 0x7f645800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646813e570 con 0x7f6458004d90 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458005400 con 0x7f6458003fb0 2023-11-24T07:59:40.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580055c0 con 0x7f64680cff10 2023-11-24T07:59:40.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6458004d90 0x7f645800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f6467fff700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6458003fb0 0x7f6458004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646ca9b700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 0x7f6458003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 -- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 msgr2=0x7f6458003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64680cff10 0x7f6458003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 -- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6458003fb0 msgr2=0x7f6458004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6458003fb0 0x7f6458004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 -- 172.21.15.32:0/1815774007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580035b0 con 0x7f6458004d90 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f646d29c700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6458004d90 0x7f645800ba10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6460009fc0 tx=0x7f6458005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df913fac7395232d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000ec30 con 0x7f6458004d90 2023-11-24T07:59:40.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646000f040 con 0x7f6458004d90 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.665+0000 7f64677fe700 1 -- 172.21.15.32:0/1815774007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460014480 con 0x7f6458004d90 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646ecff700 1 -- 172.21.15.32:0/1815774007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6458004d90 msgr2=0x7f645800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646ecff700 1 --2- 172.21.15.32:0/1815774007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6458004d90 0x7f645800ba10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6460009fc0 tx=0x7f6458005880 comp rx=0 tx=0).stop 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646d29c700 1 -- 172.21.15.32:0/1815774007 reap_dead start 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646ecff700 1 -- 172.21.15.32:0/1815774007 shutdown_connections 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646ecff700 1 -- 172.21.15.32:0/1815774007 >> 172.21.15.32:0/1815774007 conn(0x7f6468072990 msgr2=0x7f646806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646ecff700 1 -- 172.21.15.32:0/1815774007 shutdown_connections 2023-11-24T07:59:40.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.666+0000 7f646ecff700 1 -- 172.21.15.32:0/1815774007 wait complete. 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.667+0000 7f646ecff700 1 Processor -- start 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.667+0000 7f646ecff700 1 -- start start 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ecff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 0x7f64681eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ecff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64681f30d0 0x7f646815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ecff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646815a580 0x7f646813e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ecff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646815aa30 con 0x7f64681f2cf0 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ecff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646813d8d0 con 0x7f646815a580 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ecff700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646813f2c0 con 0x7f64681f30d0 2023-11-24T07:59:40.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 0x7f64681eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 0x7f64681eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58500/0 (socket says 172.21.15.32:58500) 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 -- 172.21.15.32:0/651469804 learned_addr learned my addr 172.21.15.32:0/651469804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f6467fff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64681f30d0 0x7f646815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 -- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64681f30d0 msgr2=0x7f646815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64681f30d0 0x7f646815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 -- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646815a580 msgr2=0x7f646813e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646815a580 0x7f646813e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.668+0000 7f646ca9b700 1 -- 172.21.15.32:0/651469804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646813f5a0 con 0x7f64681f2cf0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.669+0000 7f646ca9b700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 0x7f64681eff70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f645c00f7f0 tx=0x7f645c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.669+0000 7f64657fa700 1 -- 172.21.15.32:0/651469804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0052e0 con 0x7f64681f2cf0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.669+0000 7f64657fa700 1 -- 172.21.15.32:0/651469804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c005440 con 0x7f64681f2cf0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.669+0000 7f64657fa700 1 -- 172.21.15.32:0/651469804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0038c0 con 0x7f64681f2cf0 2023-11-24T07:59:40.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.669+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64681429b0 con 0x7f64681f2cf0 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.669+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6468142f70 con 0x7f64681f2cf0 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.670+0000 7f64657fa700 1 -- 172.21.15.32:0/651469804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f645c003a20 con 0x7f64681f2cf0 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.671+0000 7f64657fa700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f644c06aed0 0x7f644c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.671+0000 7f64657fa700 1 -- 172.21.15.32:0/651469804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f645c0893d0 con 0x7f64681f2cf0 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.671+0000 7f6467fff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f644c06aed0 0x7f644c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.671+0000 7f6467fff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f644c06aed0 0x7f644c06d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64540048d0 tx=0x7f6454009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.672+0000 7f647aef8b80 1 Processor -- start 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.672+0000 7f647aef8b80 1 -- start start 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed04840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecd3ed0 0x558f5ed04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 0x558f5ece7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f647aef8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ec31770 con 0x558f5ecbb490 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ec18750 con 0x558f5ecd3ed0 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f5ec16070 con 0x558f5ecbe080 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 0x558f5ece7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 0x558f5ece7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58512/0 (socket says 172.21.15.32:58512) 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f646d29c700 1 -- 172.21.15.32:0/1901173554 learned_addr learned my addr 172.21.15.32:0/1901173554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f646ca9b700 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed04840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.673+0000 7f6467fff700 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecd3ed0 0x558f5ed04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f646d29c700 1 -- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 msgr2=0x558f5ed04840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f646d29c700 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed04840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f646d29c700 1 -- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecd3ed0 msgr2=0x558f5ed04d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f646d29c700 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecd3ed0 0x558f5ed04d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f646d29c700 1 -- 172.21.15.32:0/1901173554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5ea8e5e0 con 0x558f5ecbb490 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f646d29c700 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 0x558f5ece7e70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6460016000 tx=0x7f6460004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d3a883806dcbba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f64527fc700 1 -- 172.21.15.32:0/1901173554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460007000 con 0x558f5ecbb490 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f64527fc700 1 -- 172.21.15.32:0/1901173554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646000ec30 con 0x558f5ecbb490 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f64527fc700 1 -- 172.21.15.32:0/1901173554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460014480 con 0x558f5ecbb490 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f647aef8b80 1 -- 172.21.15.32:0/1901173554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 msgr2=0x558f5ece7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.674+0000 7f647aef8b80 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 0x558f5ece7e70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6460016000 tx=0x7f6460004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 -- 172.21.15.32:0/1901173554 shutdown_connections 2023-11-24T07:59:40.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecbb490 0x558f5ece7e70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed04840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 --2- 172.21.15.32:0/1901173554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecd3ed0 0x558f5ed04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 -- 172.21.15.32:0/1901173554 >> 172.21.15.32:0/1901173554 conn(0x558f5ebec310 msgr2=0x558f5ed02e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 -- 172.21.15.32:0/1901173554 shutdown_connections 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 -- 172.21.15.32:0/1901173554 wait complete. 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 -- start start 2023-11-24T07:59:40.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.675+0000 7f647aef8b80 1 Processor -- start 2023-11-24T07:59:40.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ecd4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ecb2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ecb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ec11960 con 0x558f5ecd3ed0 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ebcb510 con 0x558f5ecbb490 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f5ebcb1b0 con 0x558f5ecbe080 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f647aef8b80 10 client.0 _ll_register_callbacks cb 0x558f5eaf6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ecb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ecb2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58518/0 (socket says 172.21.15.32:58518) 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f646d29c700 1 -- 172.21.15.32:0/3332959990 learned_addr learned my addr 172.21.15.32:0/3332959990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f6467fff700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ecb2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.676+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ecd4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f646d29c700 1 -- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 msgr2=0x558f5ecb2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f646d29c700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ecb2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f646d29c700 1 -- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 msgr2=0x558f5ecd4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f646d29c700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ecd4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f646d29c700 1 -- 172.21.15.32:0/3332959990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64580035b0 con 0x558f5ecd3ed0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f646d29c700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ecb2870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f64580053d0 tx=0x7f6460004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1741a2dd4f50c609 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f64477fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f64477fe700 1 -- 172.21.15.32:0/3332959990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460004040 con 0x558f5ecd3ed0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f64477fe700 1 -- 172.21.15.32:0/3332959990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64600041a0 con 0x558f5ecd3ed0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f5ecb0a40 con 0x558f5ecd3ed0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f64477fe700 1 -- 172.21.15.32:0/3332959990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6460004360 con 0x558f5ecd3ed0 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 10 client.5274 fetch_fsmap learned FSMap version 16 2023-11-24T07:59:40.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558f5ea3ad90 con 0x558f5ecd3ed0 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f64477fe700 1 -- 172.21.15.32:0/3332959990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f5ecb0a40 con 0x558f5ecd3ed0 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f64477fe700 1 -- 172.21.15.32:0/3332959990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f64600044c0 con 0x558f5ecd3ed0 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 10 client.5274 fetch_fsmap finished waiting for FSMap version 16 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 10 client.5274 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 20 client.5274 populate_metadata read hostname 'smithi032' 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ecff3e0 0x558f5ecff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 4 client.5274 mds_command: new command op to 5106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.677+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 --> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f5ec8c840 con 0x558f5ecff3e0 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.678+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ecff3e0 0x558f5ecff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.678+0000 7f64477fe700 10 client.5274 ms_handle_connect on v2:172.21.15.32:6833/598023464 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.678+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ecff3e0 0x558f5ecff7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07af0882ca285a6 server_cookie=d23850f795b9762a in_seq=0 out_seq=0 2023-11-24T07:59:40.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.678+0000 7f64477fe700 1 -- 172.21.15.32:0/3332959990 <== mds.0 v2:172.21.15.32:6833/598023464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f5ec8c840 con 0x558f5ecff3e0 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.678+0000 7f64477fe700 10 client.5274 handle_command_reply: tid=0 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.678+0000 7f647aef8b80 1 client.5274 shutdown 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 msgr2=0x558f5ecb2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ecb2870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f64580053d0 tx=0x7f6460004b40 comp rx=0 tx=0).stop 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 shutdown_connections 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ecff3e0 0x558f5ecff7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ecb2870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f64477fe700 0 client.5274 ms_handle_reset on v2:172.21.15.32:6833/598023464 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ecd4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3332959990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ecb2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 >> 172.21.15.32:0/3332959990 conn(0x558f5ebec310 msgr2=0x558f5ed01300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 shutdown_connections 2023-11-24T07:59:40.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 1 -- 172.21.15.32:0/3332959990 wait complete. 2023-11-24T07:59:40.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.679+0000 7f647aef8b80 20 client.5274 trim_cache size 0 max 16384 2023-11-24T07:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.697+0000 7f647aef8b80 1 Processor -- start 2023-11-24T07:59:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.697+0000 7f647aef8b80 1 -- start start 2023-11-24T07:59:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ed06930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f647aef8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ea908d0 con 0x558f5ecd3ed0 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ecba3d0 con 0x558f5ecbb490 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f5ed03180 con 0x558f5ecbe080 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ed06930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ed06930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58526/0 (socket says 172.21.15.32:58526) 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 -- 172.21.15.32:0/3043309769 learned_addr learned my addr 172.21.15.32:0/3043309769 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f6467fff700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 -- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 msgr2=0x558f5ed063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 -- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 msgr2=0x558f5ed02c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed02c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646d29c700 1 -- 172.21.15.32:0/3043309769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5ea23c10 con 0x558f5ecd3ed0 2023-11-24T07:59:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f6467fff700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.698+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f646d29c700 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ed06930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6460000f80 tx=0x7f646002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee43e3dd1d3dbb6b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f6452ffd700 1 -- 172.21.15.32:0/3043309769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460007000 con 0x558f5ecd3ed0 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f6452ffd700 1 -- 172.21.15.32:0/3043309769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6460003f80 con 0x558f5ecd3ed0 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f6452ffd700 1 -- 172.21.15.32:0/3043309769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460004310 con 0x558f5ecd3ed0 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 -- 172.21.15.32:0/3043309769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 msgr2=0x558f5ed06930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ed06930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6460000f80 tx=0x7f646002d000 comp rx=0 tx=0).stop 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 -- 172.21.15.32:0/3043309769 shutdown_connections 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ed06930 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3043309769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 -- 172.21.15.32:0/3043309769 >> 172.21.15.32:0/3043309769 conn(0x558f5ebec310 msgr2=0x558f5ece8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 -- 172.21.15.32:0/3043309769 shutdown_connections 2023-11-24T07:59:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.699+0000 7f647aef8b80 1 -- 172.21.15.32:0/3043309769 wait complete. 2023-11-24T07:59:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.700+0000 7f647aef8b80 1 -- start start 2023-11-24T07:59:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.700+0000 7f647aef8b80 1 Processor -- start 2023-11-24T07:59:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ece6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ed14650 con 0x558f5ecd3ed0 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5ed14ab0 con 0x558f5ecbb490 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f5ecfc910 con 0x558f5ecbe080 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f647aef8b80 10 client.0 _ll_register_callbacks cb 0x7f64600095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ece6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ece6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58532/0 (socket says 172.21.15.32:58532) 2023-11-24T07:59:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 -- 172.21.15.32:0/3172420215 learned_addr learned my addr 172.21.15.32:0/3172420215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f6467fff700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 -- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 msgr2=0x558f5ed14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed14110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 -- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 msgr2=0x558f5ed6dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed6dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 -- 172.21.15.32:0/3172420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f5ea3ad90 con 0x558f5ecd3ed0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed6dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f646d29c700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ece6d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f646002d040 tx=0x7f6460020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4981f728c3a6fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.701+0000 7f64477fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f64477fe700 1 -- 172.21.15.32:0/3172420215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64600042f0 con 0x558f5ecd3ed0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f64477fe700 1 -- 172.21.15.32:0/3172420215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6460046030 con 0x558f5ecd3ed0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f646000ec30 con 0x558f5ecd3ed0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f64477fe700 1 -- 172.21.15.32:0/3172420215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6460004450 con 0x558f5ecd3ed0 2023-11-24T07:59:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 10 client.5280 fetch_fsmap learned FSMap version 16 2023-11-24T07:59:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558f5ea61200 con 0x558f5ecd3ed0 2023-11-24T07:59:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f64477fe700 1 -- 172.21.15.32:0/3172420215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646000ec30 con 0x558f5ecd3ed0 2023-11-24T07:59:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f64477fe700 1 -- 172.21.15.32:0/3172420215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6460014480 con 0x558f5ecd3ed0 2023-11-24T07:59:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 10 client.5280 fetch_fsmap finished waiting for FSMap version 16 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 10 client.5280 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 20 client.5280 populate_metadata read hostname 'smithi032' 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ed06300 0x558f5ed066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 4 client.5280 mds_command: new command op to 5106 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 --> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f5ebd7f80 con 0x558f5ed06300 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ed06300 0x558f5ed066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f64477fe700 10 client.5280 ms_handle_connect on v2:172.21.15.32:6833/598023464 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.702+0000 7f646ca9b700 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ed06300 0x558f5ed066e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8427013628e74eb5 server_cookie=3b7db12983fe32d1 in_seq=0 out_seq=0 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f64477fe700 1 -- 172.21.15.32:0/3172420215 <== mds.0 v2:172.21.15.32:6833/598023464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x558f5ebd7f80 con 0x558f5ed06300 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f64477fe700 10 client.5280 handle_command_reply: tid=0 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 client.5280 shutdown 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 msgr2=0x558f5ece6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ece6d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f646002d040 tx=0x7f6460020000 comp rx=0 tx=0).stop 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 shutdown_connections 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:6833/598023464,v1:172.21.15.32:6835/598023464] conn(0x558f5ed06300 0x558f5ed066e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558f5ecd3ed0 0x558f5ece6d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f64477fe700 0 client.5280 ms_handle_reset on v2:172.21.15.32:6833/598023464 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f5ecbb490 0x558f5ed6dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 --2- 172.21.15.32:0/3172420215 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x558f5ecbe080 0x558f5ed14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 >> 172.21.15.32:0/3172420215 conn(0x558f5ebec310 msgr2=0x558f5ecb8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 shutdown_connections 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 1 -- 172.21.15.32:0/3172420215 wait complete. 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.703+0000 7f647aef8b80 20 client.5280 trim_cache size 0 max 16384 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "0b3006fa-b594-4439-bcd2-bb19d4a76534", 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5106, 2023-11-24T07:59:40.715 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 6.015723725, 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 16, 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 32, 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 32, 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 13.885518915 2023-11-24T07:59:40.716 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T07:59:40.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f644c06aed0 msgr2=0x7f644c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f644c06aed0 0x7f644c06d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64540048d0 tx=0x7f6454009040 comp rx=0 tx=0).stop 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 msgr2=0x7f64681eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 0x7f64681eff70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f645c00f7f0 tx=0x7f645c00fbb0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 shutdown_connections 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f644c06aed0 0x7f644c06d390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f646815a580 0x7f646813e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64681f2cf0 0x7f64681eff70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 --2- 172.21.15.32:0/651469804 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64681f30d0 0x7f646815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 >> 172.21.15.32:0/651469804 conn(0x7f6468072990 msgr2=0x7f646806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 shutdown_connections 2023-11-24T07:59:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.704+0000 7f646ecff700 1 -- 172.21.15.32:0/651469804 wait complete. 2023-11-24T07:59:40.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:59:40.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.842+0000 7fcb2cc99700 1 Processor -- start 2023-11-24T07:59:40.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.843+0000 7fcb2cc99700 1 -- start start 2023-11-24T07:59:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.843+0000 7fcb2cc99700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb2815e080 0x7fcb280b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.843+0000 7fcb2cc99700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 0x7fcb280afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2cc99700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb280ae430 con 0x7fcb2806c000 2023-11-24T07:59:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2cc99700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb280af0b0 con 0x7fcb2815e080 2023-11-24T07:59:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2cc99700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b05a0 con 0x7fcb280af4d0 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb26d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb2806c000 0x7fcb28073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60252/0 (socket says 172.21.15.32:60252) 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb26d9d700 1 -- 172.21.15.32:0/372563514 learned_addr learned my addr 172.21.15.32:0/372563514 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb25d9b700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 0x7fcb280afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2659c700 1 --2- 172.21.15.32:0/372563514 >> v2:172.21.15.123:3300/0 conn(0x7fcb2815e080 0x7fcb280b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614006486 0 0) 0x7fcb280ae430 con 0x7fcb2806c000 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb100035b0 con 0x7fcb2806c000 2023-11-24T07:59:40.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704392527 0 0) 0x7fcb100035b0 con 0x7fcb2806c000 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 msgr2=0x7fcb280afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 0x7fcb280afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 >> v2:172.21.15.123:3300/0 conn(0x7fcb2815e080 msgr2=0x7fcb280b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 --2- 172.21.15.32:0/372563514 >> v2:172.21.15.123:3300/0 conn(0x7fcb2815e080 0x7fcb280b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280ae710 con 0x7fcb2806c000 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fcb200019f0 con 0x7fcb2806c000 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 >> v1:172.21.15.32:6789/0 conn(0x7fcb2806c000 legacy=0x7fcb28073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb10003a30 0x7fcb10003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 0x7fcb10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10004d90 0x7fcb10005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280af0b0 con 0x7fcb10003a30 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b05a0 con 0x7fcb10004d90 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1000ba10 con 0x7fcb280af4d0 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.844+0000 7fcb2659c700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb10003a30 0x7fcb10003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb26d9d700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10004d90 0x7fcb10005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb25d9b700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 0x7fcb10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2659c700 1 -- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 msgr2=0x7fcb10004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2659c700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280af4d0 0x7fcb10004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2659c700 1 -- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10004d90 msgr2=0x7fcb10005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2659c700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10004d90 0x7fcb10005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2659c700 1 -- 172.21.15.32:0/372563514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100035b0 con 0x7fcb10003a30 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2659c700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb10003a30 0x7fcb10003e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009b90 tx=0x7fcb10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2806cdb310d5ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800e6c0 con 0x7fcb10003a30 2023-11-24T07:59:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1800e820 con 0x7fcb10003a30 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.845+0000 7fcb2559a700 1 -- 172.21.15.32:0/372563514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800eb90 con 0x7fcb10003a30 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb2cc99700 1 -- 172.21.15.32:0/372563514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb10003a30 msgr2=0x7fcb10003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/372563514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb10003a30 0x7fcb10003e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009b90 tx=0x7fcb10005880 comp rx=0 tx=0).stop 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb26d9d700 1 -- 172.21.15.32:0/372563514 reap_dead start 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb2cc99700 1 -- 172.21.15.32:0/372563514 shutdown_connections 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb2cc99700 1 -- 172.21.15.32:0/372563514 >> 172.21.15.32:0/372563514 conn(0x7fcb28071b90 msgr2=0x7fcb28071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb2cc99700 1 -- 172.21.15.32:0/372563514 shutdown_connections 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.846+0000 7fcb2cc99700 1 -- 172.21.15.32:0/372563514 wait complete. 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.847+0000 7fcb2cc99700 1 Processor -- start 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.847+0000 7fcb2cc99700 1 -- start start 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2cc99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10003a30 0x7fcb28158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2cc99700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb2815f530 0x7fcb281516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2cc99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 0x7fcb280b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2cc99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb28157380 con 0x7fcb281f4130 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2cc99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb28151bf0 con 0x7fcb10003a30 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2cc99700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2815f9b0 con 0x7fcb2815f530 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 0x7fcb280b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 0x7fcb280b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58552/0 (socket says 172.21.15.32:58552) 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 -- 172.21.15.32:0/453579875 learned_addr learned my addr 172.21.15.32:0/453579875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb2659c700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10003a30 0x7fcb28158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb25d9b700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb2815f530 0x7fcb281516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 -- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb2815f530 msgr2=0x7fcb281516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb2815f530 0x7fcb281516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 -- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10003a30 msgr2=0x7fcb28158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10003a30 0x7fcb28158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 -- 172.21.15.32:0/453579875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28159560 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb25d9b700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb2815f530 0x7fcb281516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.848+0000 7fcb26d9d700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 0x7fcb280b48c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcb2806dbb0 tx=0x7fcb2000a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.849+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000ad50 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.849+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb200133f0 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.849+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb200137a0 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.849+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb28159840 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.849+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb281f45b0 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.850+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb280753f0 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.850+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb20013900 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.850+0000 7fcb16ffd700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcb0806aed0 0x7fcb0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.850+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb20089e10 con 0x7fcb281f4130 2023-11-24T07:59:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.851+0000 7fcb2659c700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcb0806aed0 0x7fcb0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.851+0000 7fcb2659c700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcb0806aed0 0x7fcb0806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb180090b0 tx=0x7fcb1801f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:40.855+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb20054f80 con 0x7fcb281f4130 2023-11-24T07:59:41.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.103+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb28067f60 con 0x7fcb281f4130 2023-11-24T07:59:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.103+0000 7fcb16ffd700 1 -- 172.21.15.32:0/453579875 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb200585a0 con 0x7fcb281f4130 2023-11-24T07:59:41.105 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcb0806aed0 msgr2=0x7fcb0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcb0806aed0 0x7fcb0806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb180090b0 tx=0x7fcb1801f040 comp rx=0 tx=0).stop 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 msgr2=0x7fcb280b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 0x7fcb280b48c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcb2806dbb0 tx=0x7fcb2000a5e0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 shutdown_connections 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcb0806aed0 0x7fcb0806d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb281f4130 0x7fcb280b48c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb10003a30 0x7fcb28158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 --2- 172.21.15.32:0/453579875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb2815f530 0x7fcb281516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 >> 172.21.15.32:0/453579875 conn(0x7fcb28071b90 msgr2=0x7fcb2806eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 shutdown_connections 2023-11-24T07:59:41.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:41.105+0000 7fcb2cc99700 1 -- 172.21.15.32:0/453579875 wait complete. 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:59:41.119 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:59:41.120 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:59:41.120 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:59:41.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.120 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-11-24T07:59:41.138 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:41.138 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:59:41.139 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42334sec preferred_lft 42334sec 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:82:7e:02:41:a6 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::b2:e9ff:fe75:3b63/64 scope link 2023-11-24T07:59:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 82:17:03:92:a2:96 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::8017:3ff:fe92:a296/64 scope link 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:82:7e:02:41:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::7082:7eff:fe02:41a6/64 scope link 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:41.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.141 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:59:41.207 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-11-24T07:59:41.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.207 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-11-24T07:59:41.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:41.266 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-11-24T07:59:41.266 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:59:41.266 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:59:41.266 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:59:41.266 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:59:41.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.266 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-11-24T07:59:41.267 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-11-24T07:59:41.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:59:41.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:59:41.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:59:41.267 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-11-24T07:59:41.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:41.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.373 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T07:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T07:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:59:41.432 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:59:41.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-11-24T07:59:41.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:41.500 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:41.557 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:59:41.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:41.558 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:41.614 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:59:41.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:41.702 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:41.702 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:41.732 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:41.733 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:41.806 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:41.807 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:41.807 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-11-24T07:59:41.807 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-11-24T07:59:41.807 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0b3006fa-b594-4439-bcd2-bb19d4a76534" will pass to kernel 2023-11-24T07:59:41.808 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:59:41.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:41.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:41.836 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout:client5092 2023-11-24T07:59:41.918 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:41.918 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:41.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:59:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.138+0000 7f9e1daa5700 1 Processor -- start 2023-11-24T07:59:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.138+0000 7f9e1daa5700 1 -- start start 2023-11-24T07:59:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e1daa5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1815bdd0 0x7f9e181604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e1daa5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9e180b3370 0x7f9e180b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e1daa5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180b2030 con 0x7f9e180d1c70 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e1daa5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180b2f50 con 0x7f9e1815bdd0 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e1daa5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180b22d0 con 0x7f9e180b3370 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1815bdd0 0x7f9e181604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1815bdd0 0x7f9e181604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58560/0 (socket says 172.21.15.32:58560) 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.139+0000 7f9e177fe700 1 -- 172.21.15.32:0/3487982072 learned_addr learned my addr 172.21.15.32:0/3487982072 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e16ffd700 1 --2- 172.21.15.32:0/3487982072 >> v2:172.21.15.123:3300/0 conn(0x7f9e180b3370 0x7f9e180b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e177fe700 1 -- 172.21.15.32:0/3487982072 >> v2:172.21.15.123:3300/0 conn(0x7f9e180b3370 msgr2=0x7f9e180b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e177fe700 1 --2- 172.21.15.32:0/3487982072 >> v2:172.21.15.123:3300/0 conn(0x7f9e180b3370 0x7f9e180b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e177fe700 1 -- 172.21.15.32:0/3487982072 >> v1:172.21.15.32:6789/0 conn(0x7f9e180d1c70 legacy=0x7f9e180ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e177fe700 1 -- 172.21.15.32:0/3487982072 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180b25b0 con 0x7f9e1815bdd0 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e177fe700 1 --2- 172.21.15.32:0/3487982072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1815bdd0 0x7f9e181604e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e180714c0 tx=0x7f9e08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360463fd5c705a7c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e167fc700 1 -- 172.21.15.32:0/3487982072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800b870 con 0x7f9e1815bdd0 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e167fc700 1 -- 172.21.15.32:0/3487982072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800b9d0 con 0x7f9e1815bdd0 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.140+0000 7f9e167fc700 1 -- 172.21.15.32:0/3487982072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800bd60 con 0x7f9e1815bdd0 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 -- 172.21.15.32:0/3487982072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1815bdd0 msgr2=0x7f9e181604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/3487982072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1815bdd0 0x7f9e181604e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e180714c0 tx=0x7f9e08009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 -- 172.21.15.32:0/3487982072 shutdown_connections 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/3487982072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1815bdd0 0x7f9e181604e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/3487982072 >> v2:172.21.15.123:3300/0 conn(0x7f9e180b3370 0x7f9e180b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 -- 172.21.15.32:0/3487982072 >> 172.21.15.32:0/3487982072 conn(0x7f9e180731a0 msgr2=0x7f9e1809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 -- 172.21.15.32:0/3487982072 shutdown_connections 2023-11-24T07:59:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.141+0000 7f9e1daa5700 1 -- 172.21.15.32:0/3487982072 wait complete. 2023-11-24T07:59:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.142+0000 7f9e1daa5700 1 Processor -- start 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.142+0000 7f9e1daa5700 1 -- start start 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e1daa5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 0x7f9e1806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e1daa5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e180d1c70 0x7f9e1806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e1daa5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1815bdd0 0x7f9e18157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e1daa5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1806c720 con 0x7f9e180b3370 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e1daa5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1806b640 con 0x7f9e1815bdd0 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e1daa5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1806b920 con 0x7f9e180d1c70 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 0x7f9e1806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 0x7f9e1806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58566/0 (socket says 172.21.15.32:58566) 2023-11-24T07:59:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 -- 172.21.15.32:0/2270413385 learned_addr learned my addr 172.21.15.32:0/2270413385 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e16ffd700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e180d1c70 0x7f9e1806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 -- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e180d1c70 msgr2=0x7f9e1806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e17fff700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1815bdd0 0x7f9e18157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e180d1c70 0x7f9e1806cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 -- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1815bdd0 msgr2=0x7f9e18157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1815bdd0 0x7f9e18157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.143+0000 7f9e177fe700 1 -- 172.21.15.32:0/2270413385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08009550 con 0x7f9e180b3370 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.144+0000 7f9e177fe700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 0x7f9e1806aff0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000c00 tx=0x7f9e08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.144+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08004520 con 0x7f9e180b3370 2023-11-24T07:59:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.144+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0801e070 con 0x7f9e180b3370 2023-11-24T07:59:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.144+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1806bc00 con 0x7f9e180b3370 2023-11-24T07:59:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.144+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080038a0 con 0x7f9e180b3370 2023-11-24T07:59:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.144+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e181e6370 con 0x7f9e180b3370 2023-11-24T07:59:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.145+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e181e6500 con 0x7f9e180b3370 2023-11-24T07:59:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.145+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e08003a00 con 0x7f9e180b3370 2023-11-24T07:59:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.146+0000 7f9e0ffff700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9df806aed0 0x7f9df806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.146+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e08020070 con 0x7f9e180b3370 2023-11-24T07:59:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.146+0000 7f9e16ffd700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9df806aed0 0x7f9df806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.147+0000 7f9e16ffd700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9df806aed0 0x7f9df806d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e180c5740 tx=0x7f9e00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.150+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e08055d10 con 0x7f9e180b3370 2023-11-24T07:59:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.397+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9e18067f60 con 0x7f9e180b3370 2023-11-24T07:59:42.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.397+0000 7f9e0ffff700 1 -- 172.21.15.32:0/2270413385 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9e08015020 con 0x7f9e180b3370 2023-11-24T07:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:59:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9df806aed0 msgr2=0x7f9df806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9df806aed0 0x7f9df806d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e180c5740 tx=0x7f9e00008040 comp rx=0 tx=0).stop 2023-11-24T07:59:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 msgr2=0x7f9e1806aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 0x7f9e1806aff0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000c00 tx=0x7f9e08004750 comp rx=0 tx=0).stop 2023-11-24T07:59:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 shutdown_connections 2023-11-24T07:59:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9df806aed0 0x7f9df806d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1815bdd0 0x7f9e18157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e180b3370 0x7f9e1806aff0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 --2- 172.21.15.32:0/2270413385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e180d1c70 0x7f9e1806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 >> 172.21.15.32:0/2270413385 conn(0x7f9e180731a0 msgr2=0x7f9e180afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 shutdown_connections 2023-11-24T07:59:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:42.399+0000 7f9e1daa5700 1 -- 172.21.15.32:0/2270413385 wait complete. 2023-11-24T07:59:42.413 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:59:42.413 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:59:42.413 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-11-24T07:59:42.414 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:59:42.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:42.415 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:59:42.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:59:42.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:59:42.415 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:59:42.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.415 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-11-24T07:59:42.436 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42341sec preferred_lft 42341sec 2023-11-24T07:59:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5e64:37:eb07:4486/64 scope link noprefixroute 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5e:8e:67:5b:95:d2 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::890:f0ff:fe2b:8732/64 scope link 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5e:8e:67:5b:95:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5c8e:67ff:fe5b:95d2/64 scope link 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether f6:ae:97:48:1a:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:59:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::f4ae:97ff:fe48:1a19/64 scope link 2023-11-24T07:59:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:42.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.439 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:59:42.507 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-11-24T07:59:42.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.509 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-11-24T07:59:42.572 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-11-24T07:59:42.572 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:59:42.573 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:59:42.573 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:59:42.573 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:59:42.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.573 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-11-24T07:59:42.573 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-11-24T07:59:42.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:59:42.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:59:42.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:59:42.573 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-11-24T07:59:42.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.695 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T07:59:42.717 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T07:59:42.717 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:59:42.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:59:42.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-11-24T07:59:42.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:42.790 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:42.849 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:59:42.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:42.849 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:42.907 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:59:42.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:42.908 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:42.996 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:42.997 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:43.022 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:43.023 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:43.023 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:59:43.023 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 1@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:59:43.023 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:59:43.023 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:43.023 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:43.024 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:43.053 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:43.053 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:43.053 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-11-24T07:59:43.054 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 1@0b3006fa-b594-4439-bcd2-bb19d4a76534.cephfs=/ 2023-11-24T07:59:43.054 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-11-24T07:59:43.054 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:43.054 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:43.054 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:43.054 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-11-24T07:59:43.055 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-11-24T07:59:43.055 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0b3006fa-b594-4439-bcd2-bb19d4a76534" will pass to kernel 2023-11-24T07:59:43.056 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:59:43.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:43.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:43.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:43.086 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:43.158 INFO:teuthology.orchestra.run.smithi123.stdout:client4841 2023-11-24T07:59:43.159 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:43.159 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:43.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T07:59:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.358+0000 7fd07d328700 1 Processor -- start 2023-11-24T07:59:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.358+0000 7fd07d328700 1 -- start start 2023-11-24T07:59:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd07d328700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd07815e3c0 0x7fd0780b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd07d328700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780b6280 con 0x7fd0780b5e70 2023-11-24T07:59:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd07d328700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780d1e60 con 0x7fd07815dfe0 2023-11-24T07:59:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd07d328700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780d2ae0 con 0x7fd07815e3c0 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0767fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd07815e3c0 0x7fd0780b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0767fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd07815e3c0 0x7fd0780b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58576/0 (socket says 172.21.15.32:58576) 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0767fc700 1 -- 172.21.15.32:0/1974462287 learned_addr learned my addr 172.21.15.32:0/1974462287 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0777fe700 1 --1- 172.21.15.32:0/1974462287 >> v1:172.21.15.32:6789/0 conn(0x7fd0780b5e70 0x7fd0780b6f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60290/0 (socket says 172.21.15.32:60290) 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0767fc700 1 -- 172.21.15.32:0/1974462287 >> v1:172.21.15.123:6789/0 conn(0x7fd07815dfe0 legacy=0x7fd0780d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0767fc700 1 -- 172.21.15.32:0/1974462287 >> v1:172.21.15.32:6789/0 conn(0x7fd0780b5e70 legacy=0x7fd0780b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.359+0000 7fd0767fc700 1 -- 172.21.15.32:0/1974462287 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780d2140 con 0x7fd07815e3c0 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd0767fc700 1 --2- 172.21.15.32:0/1974462287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815e3c0 0x7fd0780b5930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098f0 tx=0x7fd068009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96db208bbd2dcff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd075ffb700 1 -- 172.21.15.32:0/1974462287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800b8a0 con 0x7fd07815e3c0 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd075ffb700 1 -- 172.21.15.32:0/1974462287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06800ba00 con 0x7fd07815e3c0 2023-11-24T07:59:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd075ffb700 1 -- 172.21.15.32:0/1974462287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800bd90 con 0x7fd07815e3c0 2023-11-24T07:59:43.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 -- 172.21.15.32:0/1974462287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815e3c0 msgr2=0x7fd0780b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 --2- 172.21.15.32:0/1974462287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815e3c0 0x7fd0780b5930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098f0 tx=0x7fd068009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:43.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 -- 172.21.15.32:0/1974462287 shutdown_connections 2023-11-24T07:59:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 --2- 172.21.15.32:0/1974462287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815e3c0 0x7fd0780b5930 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 -- 172.21.15.32:0/1974462287 >> 172.21.15.32:0/1974462287 conn(0x7fd0780d0b10 msgr2=0x7fd0780d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 -- 172.21.15.32:0/1974462287 shutdown_connections 2023-11-24T07:59:43.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.360+0000 7fd07d328700 1 -- 172.21.15.32:0/1974462287 wait complete. 2023-11-24T07:59:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.362+0000 7fd07d328700 1 Processor -- start 2023-11-24T07:59:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.362+0000 7fd07d328700 1 -- start start 2023-11-24T07:59:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.362+0000 7fd07d328700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0780b5e70 0x7fd07815e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 0x7fd0780cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd07815e3c0 0x7fd0780cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815fda0 con 0x7fd07815dfe0 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815f190 con 0x7fd0780b5e70 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815f470 con 0x7fd07815e3c0 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 0x7fd0780cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 0x7fd0780cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58582/0 (socket says 172.21.15.32:58582) 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 -- 172.21.15.32:0/2569458458 learned_addr learned my addr 172.21.15.32:0/2569458458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd076ffd700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0780b5e70 0x7fd07815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0777fe700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd07815e3c0 0x7fd0780cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 -- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd07815e3c0 msgr2=0x7fd0780cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd07815e3c0 0x7fd0780cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 -- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0780b5e70 msgr2=0x7fd07815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0780b5e70 0x7fd07815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 -- 172.21.15.32:0/2569458458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068009550 con 0x7fd07815dfe0 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd0767fc700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 0x7fd0780cd3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd068009130 tx=0x7fd068004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680036a0 con 0x7fd07815dfe0 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06801e070 con 0x7fd07815dfe0 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07815f750 con 0x7fd07815dfe0 2023-11-24T07:59:43.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068003a40 con 0x7fd07815dfe0 2023-11-24T07:59:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.363+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07815ed60 con 0x7fd07815dfe0 2023-11-24T07:59:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.364+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07815ff50 con 0x7fd07815dfe0 2023-11-24T07:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.365+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd068003ba0 con 0x7fd07815dfe0 2023-11-24T07:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.365+0000 7fd06f7fe700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd05806aed0 0x7fd05806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.365+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd068020070 con 0x7fd07815dfe0 2023-11-24T07:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.365+0000 7fd076ffd700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd05806aed0 0x7fd05806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.366+0000 7fd076ffd700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd05806aed0 0x7fd05806d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd060007fd0 tx=0x7fd060009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.370+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd07815ff50 con 0x7fd07815dfe0 2023-11-24T07:59:43.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.618+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd07815eef0 con 0x7fd07815dfe0 2023-11-24T07:59:43.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.735+0000 7fd06f7fe700 1 -- 172.21.15.32:0/2569458458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd068055ab0 con 0x7fd07815dfe0 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd05806aed0 msgr2=0x7fd05806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd05806aed0 0x7fd05806d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd060007fd0 tx=0x7fd060009040 comp rx=0 tx=0).stop 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 msgr2=0x7fd0780cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 0x7fd0780cd3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd068009130 tx=0x7fd068004750 comp rx=0 tx=0).stop 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 shutdown_connections 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd05806aed0 0x7fd05806d390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd07815e3c0 0x7fd0780cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0780b5e70 0x7fd07815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 --2- 172.21.15.32:0/2569458458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd07815dfe0 0x7fd0780cd3e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 >> 172.21.15.32:0/2569458458 conn(0x7fd0780d0b10 msgr2=0x7fd0780c99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 shutdown_connections 2023-11-24T07:59:43.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.737+0000 7fd07d328700 1 -- 172.21.15.32:0/2569458458 wait complete. 2023-11-24T07:59:43.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T07:59:43.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.876+0000 7fade8a0e700 1 Processor -- start 2023-11-24T07:59:43.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.876+0000 7fade8a0e700 1 -- start start 2023-11-24T07:59:43.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade8a0e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade4152cc0 0x7fade406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade8a0e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fade40d0630 0x7fade406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade8a0e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade415c4b0 con 0x7fade40b49a0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade8a0e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade415c790 con 0x7fade4152cc0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade8a0e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade415ca70 con 0x7fade40d0630 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade4152cc0 0x7fade406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade4152cc0 0x7fade406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58588/0 (socket says 172.21.15.32:58588) 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 -- 172.21.15.32:0/34410090 learned_addr learned my addr 172.21.15.32:0/34410090 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade2d9d700 1 --2- 172.21.15.32:0/34410090 >> v2:172.21.15.123:3300/0 conn(0x7fade40d0630 0x7fade406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 -- 172.21.15.32:0/34410090 >> v2:172.21.15.123:3300/0 conn(0x7fade40d0630 msgr2=0x7fade406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 --2- 172.21.15.32:0/34410090 >> v2:172.21.15.123:3300/0 conn(0x7fade40d0630 0x7fade406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 -- 172.21.15.32:0/34410090 >> v1:172.21.15.32:6789/0 conn(0x7fade40b49a0 legacy=0x7fade40cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 -- 172.21.15.32:0/34410090 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade415cd50 con 0x7fade4152cc0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade259c700 1 --2- 172.21.15.32:0/34410090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4152cc0 0x7fade406c000 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fade40b4e80 tx=0x7fadd8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31be171ea6ffa66 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.877+0000 7fade159a700 1 -- 172.21.15.32:0/34410090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800b870 con 0x7fade4152cc0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade159a700 1 -- 172.21.15.32:0/34410090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800b9d0 con 0x7fade4152cc0 2023-11-24T07:59:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade159a700 1 -- 172.21.15.32:0/34410090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800bd60 con 0x7fade4152cc0 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 -- 172.21.15.32:0/34410090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4152cc0 msgr2=0x7fade406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 --2- 172.21.15.32:0/34410090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4152cc0 0x7fade406c000 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fade40b4e80 tx=0x7fadd8009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 -- 172.21.15.32:0/34410090 shutdown_connections 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 --2- 172.21.15.32:0/34410090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4152cc0 0x7fade406c000 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 --2- 172.21.15.32:0/34410090 >> v2:172.21.15.123:3300/0 conn(0x7fade40d0630 0x7fade406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 -- 172.21.15.32:0/34410090 >> 172.21.15.32:0/34410090 conn(0x7fade4071b90 msgr2=0x7fade4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 -- 172.21.15.32:0/34410090 shutdown_connections 2023-11-24T07:59:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.878+0000 7fade8a0e700 1 -- 172.21.15.32:0/34410090 wait complete. 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.879+0000 7fade8a0e700 1 Processor -- start 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 -- start start 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade40b49a0 0x7fade4158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 0x7fade40b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fade4152cc0 0x7fade40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade414ed90 con 0x7fade40d0630 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41537b0 con 0x7fade40b49a0 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade8a0e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4153a90 con 0x7fade4152cc0 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 0x7fade40b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 0x7fade40b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58596/0 (socket says 172.21.15.32:58596) 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.880+0000 7fade1d9b700 1 -- 172.21.15.32:0/119479287 learned_addr learned my addr 172.21.15.32:0/119479287 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade259c700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade40b49a0 0x7fade4158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade2d9d700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fade4152cc0 0x7fade40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade1d9b700 1 -- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fade4152cc0 msgr2=0x7fade40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade1d9b700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fade4152cc0 0x7fade40cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade1d9b700 1 -- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade40b49a0 msgr2=0x7fade4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade1d9b700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade40b49a0 0x7fade4158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade1d9b700 1 -- 172.21.15.32:0/119479287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8009550 con 0x7fade40d0630 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade259c700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade40b49a0 0x7fade4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade2d9d700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fade4152cc0 0x7fade40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade1d9b700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 0x7fade40b1f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c4e0 tx=0x7fadd400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40052d0 con 0x7fade40d0630 2023-11-24T07:59:43.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd4005430 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade4153d70 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40038c0 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.881+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade4154050 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.882+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade40753f0 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.882+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadd4003a20 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.883+0000 7fadd2ffd700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fadc406aed0 0x7fadc406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.883+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd4088ab0 con 0x7fade40d0630 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.883+0000 7fade259c700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fadc406aed0 0x7fadc406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.884+0000 7fade259c700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fadc406aed0 0x7fadc406d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fade406b240 tx=0x7fadd8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:43.888+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadd4053b60 con 0x7fade40d0630 2023-11-24T07:59:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.117+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fade4067f60 con 0x7fadc406aed0 2023-11-24T07:59:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.119+0000 7fadd2ffd700 1 -- 172.21.15.32:0/119479287 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fade4067f60 con 0x7fadc406aed0 2023-11-24T07:59:44.120 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-11-24T07:59:44.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T07:59:44.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-11-24T07:59:44.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T07:59:44.121 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fadc406aed0 msgr2=0x7fadc406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fadc406aed0 0x7fadc406d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fade406b240 tx=0x7fadd8008030 comp rx=0 tx=0).stop 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 msgr2=0x7fade40b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 0x7fade40b1f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c4e0 tx=0x7fadd400c8a0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 shutdown_connections 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fadc406aed0 0x7fadc406d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fade4152cc0 0x7fade40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fade40b49a0 0x7fade4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 --2- 172.21.15.32:0/119479287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40d0630 0x7fade40b1f60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 >> 172.21.15.32:0/119479287 conn(0x7fade4071b90 msgr2=0x7fade415a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 shutdown_connections 2023-11-24T07:59:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.121+0000 7fade8a0e700 1 -- 172.21.15.32:0/119479287 wait complete. 2023-11-24T07:59:44.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:59:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.260+0000 7fa158df8700 1 Processor -- start 2023-11-24T07:59:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.260+0000 7fa158df8700 1 -- start start 2023-11-24T07:59:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa158df8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa15415f5b0 0x7fa1540cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa158df8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1540b1890 0x7fa1540ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa158df8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1540ce410 con 0x7fa1540b4330 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa158df8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540b27a0 con 0x7fa1540b1890 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa158df8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540b32a0 con 0x7fa15415f5b0 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa152d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1540b1890 0x7fa1540ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa152d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1540b1890 0x7fa1540ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58606/0 (socket says 172.21.15.32:58606) 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.261+0000 7fa152d9d700 1 -- 172.21.15.32:0/3459362728 learned_addr learned my addr 172.21.15.32:0/3459362728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa15259c700 1 --2- 172.21.15.32:0/3459362728 >> v2:172.21.15.123:3300/0 conn(0x7fa15415f5b0 0x7fa1540cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa152d9d700 1 -- 172.21.15.32:0/3459362728 >> v2:172.21.15.123:3300/0 conn(0x7fa15415f5b0 msgr2=0x7fa1540cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa152d9d700 1 --2- 172.21.15.32:0/3459362728 >> v2:172.21.15.123:3300/0 conn(0x7fa15415f5b0 0x7fa1540cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa152d9d700 1 -- 172.21.15.32:0/3459362728 >> v1:172.21.15.123:6789/0 conn(0x7fa1540b4330 legacy=0x7fa1540b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa152d9d700 1 -- 172.21.15.32:0/3459362728 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540ce0c0 con 0x7fa1540b1890 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa152d9d700 1 --2- 172.21.15.32:0/3459362728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b1890 0x7fa1540ce630 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa154075600 tx=0x7fa14400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b544038a34be89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa15159a700 1 -- 172.21.15.32:0/3459362728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400e070 con 0x7fa1540b1890 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa15159a700 1 -- 172.21.15.32:0/3459362728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144003a20 con 0x7fa1540b1890 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.262+0000 7fa15159a700 1 -- 172.21.15.32:0/3459362728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144003dd0 con 0x7fa1540b1890 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 -- 172.21.15.32:0/3459362728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b1890 msgr2=0x7fa1540ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 --2- 172.21.15.32:0/3459362728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b1890 0x7fa1540ce630 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa154075600 tx=0x7fa14400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 -- 172.21.15.32:0/3459362728 shutdown_connections 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 --2- 172.21.15.32:0/3459362728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b1890 0x7fa1540ce630 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 --2- 172.21.15.32:0/3459362728 >> v2:172.21.15.123:3300/0 conn(0x7fa15415f5b0 0x7fa1540cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 -- 172.21.15.32:0/3459362728 >> 172.21.15.32:0/3459362728 conn(0x7fa1540b3810 msgr2=0x7fa1540b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 -- 172.21.15.32:0/3459362728 shutdown_connections 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.263+0000 7fa158df8700 1 -- 172.21.15.32:0/3459362728 wait complete. 2023-11-24T07:59:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.264+0000 7fa158df8700 1 Processor -- start 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.264+0000 7fa158df8700 1 -- start start 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa158df8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1540b1890 0x7fa15406cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa158df8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 0x7fa1540b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa158df8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415f5b0 0x7fa1540b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa158df8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15415ba90 con 0x7fa1540b4330 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa158df8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1541f45e0 con 0x7fa1540b1890 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa158df8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1541f3cb0 con 0x7fa15415f5b0 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 0x7fa1540b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 0x7fa1540b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58610/0 (socket says 172.21.15.32:58610) 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 -- 172.21.15.32:0/3511321564 learned_addr learned my addr 172.21.15.32:0/3511321564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa15259c700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1540b1890 0x7fa15406cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa152d9d700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415f5b0 0x7fa1540b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 -- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415f5b0 msgr2=0x7fa1540b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415f5b0 0x7fa1540b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 -- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1540b1890 msgr2=0x7fa15406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1540b1890 0x7fa15406cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 -- 172.21.15.32:0/3511321564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa144009b30 con 0x7fa1540b4330 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa152d9d700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415f5b0 0x7fa1540b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.265+0000 7fa151d9b700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 0x7fa1540b0b00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa14800b470 tx=0x7fa14800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.266+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148011830 con 0x7fa1540b4330 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.266+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1480119b0 con 0x7fa1540b4330 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.266+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1541f3f90 con 0x7fa1540b4330 2023-11-24T07:59:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.266+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148011d60 con 0x7fa1540b4330 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.266+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1541f4a70 con 0x7fa1540b4330 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.267+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1540d12e0 con 0x7fa1540b4330 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.267+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa14801a400 con 0x7fa1540b4330 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.267+0000 7fa142ffd700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13406af00 0x7fa13406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.268+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa148089580 con 0x7fa1540b4330 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.268+0000 7fa15259c700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13406af00 0x7fa13406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.268+0000 7fa15259c700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13406af00 0x7fa13406d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa15415c2f0 tx=0x7fa14c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.272+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa148054630 con 0x7fa1540b4330 2023-11-24T07:59:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.475+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa154068fb0 con 0x7fa1540b4330 2023-11-24T07:59:44.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.476+0000 7fa142ffd700 1 -- 172.21.15.32:0/3511321564 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa148057c50 con 0x7fa1540b4330 2023-11-24T07:59:44.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13406af00 msgr2=0x7fa13406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13406af00 0x7fa13406d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa15415c2f0 tx=0x7fa14c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 msgr2=0x7fa1540b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 0x7fa1540b0b00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa14800b470 tx=0x7fa14800b780 comp rx=0 tx=0).stop 2023-11-24T07:59:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 shutdown_connections 2023-11-24T07:59:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13406af00 0x7fa13406d3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415f5b0 0x7fa1540b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1540b1890 0x7fa15406cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 --2- 172.21.15.32:0/3511321564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540b4330 0x7fa1540b0b00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 >> 172.21.15.32:0/3511321564 conn(0x7fa1540b3810 msgr2=0x7fa15415d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 shutdown_connections 2023-11-24T07:59:44.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.478+0000 7fa158df8700 1 -- 172.21.15.32:0/3511321564 wait complete. 2023-11-24T07:59:44.492 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:59:44.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:44.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:44.578 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:59:44.578 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:44.578 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:44.597 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:44.597 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:44.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.818+0000 7f1eeb6a7700 1 Processor -- start 2023-11-24T07:59:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.818+0000 7f1eeb6a7700 1 -- start start 2023-11-24T07:59:44.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1eeb6a7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ee40d2240 0x7f1ee40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1eeb6a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee415e1b0 0x7f1ee40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1eeb6a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee40b1d40 con 0x7f1ee415ddd0 2023-11-24T07:59:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1eeb6a7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee40b28c0 con 0x7f1ee40d2240 2023-11-24T07:59:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1eeb6a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee406c000 con 0x7f1ee415e1b0 2023-11-24T07:59:44.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9443700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ee415ddd0 0x7f1ee40d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60328/0 (socket says 172.21.15.32:60328) 2023-11-24T07:59:44.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9443700 1 -- 172.21.15.32:0/4144563227 learned_addr learned my addr 172.21.15.32:0/4144563227 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:44.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee8c42700 1 --2- 172.21.15.32:0/4144563227 >> v2:172.21.15.123:3300/0 conn(0x7f1ee40d2240 0x7f1ee40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9c44700 1 --2- 172.21.15.32:0/4144563227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee415e1b0 0x7f1ee40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1edbfff700 1 -- 172.21.15.32:0/4144563227 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100682622 0 0) 0x7f1ee40b1d40 con 0x7f1ee415ddd0 2023-11-24T07:59:44.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1edbfff700 1 -- 172.21.15.32:0/4144563227 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed00035b0 con 0x7f1ee415ddd0 2023-11-24T07:59:44.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1edbfff700 1 -- 172.21.15.32:0/4144563227 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038490350 0 0) 0x7f1ed00035b0 con 0x7f1ee415ddd0 2023-11-24T07:59:44.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9c44700 1 -- 172.21.15.32:0/4144563227 >> v2:172.21.15.123:3300/0 conn(0x7f1ee40d2240 msgr2=0x7f1ee40b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:44.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9c44700 1 --2- 172.21.15.32:0/4144563227 >> v2:172.21.15.123:3300/0 conn(0x7f1ee40d2240 0x7f1ee40b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9c44700 1 -- 172.21.15.32:0/4144563227 >> v1:172.21.15.32:6789/0 conn(0x7f1ee415ddd0 legacy=0x7f1ee40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee9c44700 1 -- 172.21.15.32:0/4144563227 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40b1f90 con 0x7f1ee415e1b0 2023-11-24T07:59:44.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.819+0000 7f1ee8c42700 1 --2- 172.21.15.32:0/4144563227 >> v2:172.21.15.123:3300/0 conn(0x7f1ee40d2240 0x7f1ee40b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:44.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1ee9c44700 1 --2- 172.21.15.32:0/4144563227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee415e1b0 0x7f1ee40cf000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009dc0 tx=0x7f1edc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42a99d8210d6a8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1edbfff700 1 -- 172.21.15.32:0/4144563227 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00e070 con 0x7f1ee415e1b0 2023-11-24T07:59:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1edbfff700 1 -- 172.21.15.32:0/4144563227 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc003a20 con 0x7f1ee415e1b0 2023-11-24T07:59:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1edbfff700 1 -- 172.21.15.32:0/4144563227 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0001670 con 0x7f1ee415e1b0 2023-11-24T07:59:44.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/4144563227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee415e1b0 msgr2=0x7f1ee40cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/4144563227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee415e1b0 0x7f1ee40cf000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009dc0 tx=0x7f1edc00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/4144563227 shutdown_connections 2023-11-24T07:59:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/4144563227 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee415e1b0 0x7f1ee40cf000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/4144563227 >> v2:172.21.15.123:3300/0 conn(0x7f1ee40d2240 0x7f1ee40b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/4144563227 >> 172.21.15.32:0/4144563227 conn(0x7f1ee40731a0 msgr2=0x7f1ee409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/4144563227 shutdown_connections 2023-11-24T07:59:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.820+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/4144563227 wait complete. 2023-11-24T07:59:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 Processor -- start 2023-11-24T07:59:44.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 -- start start 2023-11-24T07:59:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee40d2240 0x7f1ee409f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ee415ddd0 0x7f1ee4157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 0x7f1ee409ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee41f3ba0 con 0x7f1ee415e1b0 2023-11-24T07:59:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415ae50 con 0x7f1ee415ddd0 2023-11-24T07:59:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1eeb6a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415b130 con 0x7f1ee40d2240 2023-11-24T07:59:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.822+0000 7f1ee9c44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 0x7f1ee409ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 0x7f1ee409ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58624/0 (socket says 172.21.15.32:58624) 2023-11-24T07:59:44.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 -- 172.21.15.32:0/966752171 learned_addr learned my addr 172.21.15.32:0/966752171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee8c42700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ee415ddd0 0x7f1ee4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9443700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee40d2240 0x7f1ee409f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 -- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee40d2240 msgr2=0x7f1ee409f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee40d2240 0x7f1ee409f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 -- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ee415ddd0 msgr2=0x7f1ee4157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ee415ddd0 0x7f1ee4157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 -- 172.21.15.32:0/966752171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40094a0 con 0x7f1ee415e1b0 2023-11-24T07:59:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9443700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee40d2240 0x7f1ee409f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ee9c44700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 0x7f1ee409ccd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00b370 tx=0x7f1edc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc009450 con 0x7f1ee415e1b0 2023-11-24T07:59:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc004560 con 0x7f1ee415e1b0 2023-11-24T07:59:44.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc004910 con 0x7f1ee415e1b0 2023-11-24T07:59:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc009a70 con 0x7f1ee415e1b0 2023-11-24T07:59:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.823+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee40b2ed0 con 0x7f1ee415e1b0 2023-11-24T07:59:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.824+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee4067f60 con 0x7f1ee415e1b0 2023-11-24T07:59:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.825+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1edc012450 con 0x7f1ee415e1b0 2023-11-24T07:59:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.825+0000 7f1ed9ffb700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1ec806aed0 0x7f1ec806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.825+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1edc08abd0 con 0x7f1ee415e1b0 2023-11-24T07:59:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.825+0000 7f1ee9443700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1ec806aed0 0x7f1ec806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.826+0000 7f1ee9443700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1ec806aed0 0x7f1ec806d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00079b0 tx=0x7f1ee0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:44.830+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1edc0597a0 con 0x7f1ee415e1b0 2023-11-24T07:59:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.030+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee40b2d30 con 0x7f1ee415e1b0 2023-11-24T07:59:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.032+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/966752171 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f1ee40b2d30 con 0x7f1ee415e1b0 2023-11-24T07:59:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:45.034 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:35.687434+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1ec806aed0 msgr2=0x7f1ec806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1ec806aed0 0x7f1ec806d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00079b0 tx=0x7f1ee0009040 comp rx=0 tx=0).stop 2023-11-24T07:59:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 msgr2=0x7f1ee409ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 0x7f1ee409ccd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00b370 tx=0x7f1edc004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 shutdown_connections 2023-11-24T07:59:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f1ec806aed0 0x7f1ec806d390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ee415e1b0 0x7f1ee409ccd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ee40d2240 0x7f1ee409f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 --2- 172.21.15.32:0/966752171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ee415ddd0 0x7f1ee4157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 >> 172.21.15.32:0/966752171 conn(0x7f1ee40731a0 msgr2=0x7f1ee40cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 shutdown_connections 2023-11-24T07:59:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.034+0000 7f1eeb6a7700 1 -- 172.21.15.32:0/966752171 wait complete. 2023-11-24T07:59:45.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:32.221808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:33.067627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:45.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.175+0000 7f95217a6700 1 Processor -- start 2023-11-24T07:59:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.175+0000 7f95217a6700 1 -- start start 2023-11-24T07:59:45.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f95217a6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f95217a6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f951c0d39d0 0x7f951c0d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f95217a6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f951c0b98c0 0x7f951c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f95217a6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c15e520 con 0x7f951c0d39d0 2023-11-24T07:59:45.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f95217a6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c0d2930 con 0x7f951c0b98c0 2023-11-24T07:59:45.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f95217a6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0d2c10 con 0x7f951c15e140 2023-11-24T07:59:45.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f951c0d39d0 0x7f951c0d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f951c0d39d0 0x7f951c0d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58628/0 (socket says 172.21.15.32:58628) 2023-11-24T07:59:45.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 -- 172.21.15.32:0/4262488136 learned_addr learned my addr 172.21.15.32:0/4262488136 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951b7fe700 1 --2- 172.21.15.32:0/4262488136 >> v2:172.21.15.123:3300/0 conn(0x7f951c0b98c0 0x7f951c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951affd700 1 --2- 172.21.15.32:0/4262488136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 -- 172.21.15.32:0/4262488136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 msgr2=0x7f951c0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 --2- 172.21.15.32:0/4262488136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 -- 172.21.15.32:0/4262488136 >> v2:172.21.15.123:3300/0 conn(0x7f951c0b98c0 msgr2=0x7f951c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:45.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 --2- 172.21.15.32:0/4262488136 >> v2:172.21.15.123:3300/0 conn(0x7f951c0b98c0 0x7f951c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 -- 172.21.15.32:0/4262488136 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0d2ef0 con 0x7f951c0d39d0 2023-11-24T07:59:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f951a7fc700 1 --2- 172.21.15.32:0/4262488136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0d4400 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f951c0d4f70 tx=0x7f950c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24de251c5534b735 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.176+0000 7f9519ffb700 1 -- 172.21.15.32:0/4262488136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00b870 con 0x7f951c0d39d0 2023-11-24T07:59:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f9519ffb700 1 -- 172.21.15.32:0/4262488136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c00b9d0 con 0x7f951c0d39d0 2023-11-24T07:59:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f9519ffb700 1 -- 172.21.15.32:0/4262488136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00bd60 con 0x7f951c0d39d0 2023-11-24T07:59:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 -- 172.21.15.32:0/4262488136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 msgr2=0x7f951c0d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 --2- 172.21.15.32:0/4262488136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0d4400 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f951c0d4f70 tx=0x7f950c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 -- 172.21.15.32:0/4262488136 shutdown_connections 2023-11-24T07:59:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 --2- 172.21.15.32:0/4262488136 >> v2:172.21.15.123:3300/0 conn(0x7f951c0b98c0 0x7f951c0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 --2- 172.21.15.32:0/4262488136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 --2- 172.21.15.32:0/4262488136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0d4400 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 -- 172.21.15.32:0/4262488136 >> 172.21.15.32:0/4262488136 conn(0x7f951c06b410 msgr2=0x7f951c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 -- 172.21.15.32:0/4262488136 shutdown_connections 2023-11-24T07:59:45.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.177+0000 7f95217a6700 1 -- 172.21.15.32:0/4262488136 wait complete. 2023-11-24T07:59:45.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 Processor -- start 2023-11-24T07:59:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 -- start start 2023-11-24T07:59:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c0b98c0 0x7f951c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0c8630 con 0x7f951c0d39d0 2023-11-24T07:59:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0c7cd0 con 0x7f951c0b98c0 2023-11-24T07:59:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f95217a6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0c7f70 con 0x7f951c15e140 2023-11-24T07:59:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f951a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f951a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58634/0 (socket says 172.21.15.32:58634) 2023-11-24T07:59:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.179+0000 7f951a7fc700 1 -- 172.21.15.32:0/617450398 learned_addr learned my addr 172.21.15.32:0/617450398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:45.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951affd700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c0b98c0 0x7f951c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951b7fe700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951a7fc700 1 -- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 msgr2=0x7f951c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951a7fc700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951a7fc700 1 -- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c0b98c0 msgr2=0x7f951c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951a7fc700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c0b98c0 0x7f951c15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951a7fc700 1 -- 172.21.15.32:0/617450398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c009550 con 0x7f951c0d39d0 2023-11-24T07:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951b7fe700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f951a7fc700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0b4bc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f950c000c00 tx=0x7f950c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c004540 con 0x7f951c0d39d0 2023-11-24T07:59:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c0036a0 con 0x7f951c0d39d0 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c0c8250 con 0x7f951c0d39d0 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c003a50 con 0x7f951c0d39d0 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.180+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c0b5640 con 0x7f951c0d39d0 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.181+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c0c8770 con 0x7f951c0d39d0 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.182+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f950c021070 con 0x7f951c0d39d0 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.182+0000 7f95037fe700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f950806aed0 0x7f950806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.182+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f950c08ac50 con 0x7f951c0d39d0 2023-11-24T07:59:45.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.182+0000 7f951affd700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f950806aed0 0x7f950806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.183+0000 7f951affd700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f950806aed0 0x7f950806d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9510006b20 tx=0x7f9510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.186+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f951c0c8770 con 0x7f951c0d39d0 2023-11-24T07:59:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.385+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f951c0c8770 con 0x7f951c0d39d0 2023-11-24T07:59:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.385+0000 7f95037fe700 1 -- 172.21.15.32:0/617450398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f951c0c8770 con 0x7f951c0d39d0 2023-11-24T07:59:45.387 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f950806aed0 msgr2=0x7f950806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f950806aed0 0x7f950806d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9510006b20 tx=0x7f9510008040 comp rx=0 tx=0).stop 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 msgr2=0x7f951c0b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0b4bc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f950c000c00 tx=0x7f950c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 shutdown_connections 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f950806aed0 0x7f950806d390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f951c15e140 0x7f951c0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f951c0b98c0 0x7f951c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 --2- 172.21.15.32:0/617450398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c0d39d0 0x7f951c0b4bc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 >> 172.21.15.32:0/617450398 conn(0x7f951c06b410 msgr2=0x7f951c15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 shutdown_connections 2023-11-24T07:59:45.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.387+0000 7f95217a6700 1 -- 172.21.15.32:0/617450398 wait complete. 2023-11-24T07:59:45.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.527+0000 7f45c1d61700 1 Processor -- start 2023-11-24T07:59:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.527+0000 7f45c1d61700 1 -- start start 2023-11-24T07:59:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45c1d61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45bc15e160 0x7f45bc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45c1d61700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc0afc90 0x7f45bc073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45c1d61700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0b3c60 con 0x7f45bc0744d0 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45c1d61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0b4500 con 0x7f45bc15e160 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45c1d61700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc074960 con 0x7f45bc0afc90 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45bb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45bc15e160 0x7f45bc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45bb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45bc15e160 0x7f45bc0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58640/0 (socket says 172.21.15.32:58640) 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45bb7fe700 1 -- 172.21.15.32:0/4173687117 learned_addr learned my addr 172.21.15.32:0/4173687117 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.528+0000 7f45baffd700 1 --2- 172.21.15.32:0/4173687117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc0afc90 0x7f45bc073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45bb7fe700 1 -- 172.21.15.32:0/4173687117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc0afc90 msgr2=0x7f45bc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45bb7fe700 1 --2- 172.21.15.32:0/4173687117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc0afc90 0x7f45bc073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45bb7fe700 1 -- 172.21.15.32:0/4173687117 >> v1:172.21.15.123:6789/0 conn(0x7f45bc0744d0 legacy=0x7f45bc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45bb7fe700 1 -- 172.21.15.32:0/4173687117 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0b58c0 con 0x7f45bc15e160 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45bb7fe700 1 --2- 172.21.15.32:0/4173687117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc15e160 0x7f45bc0b5da0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098f0 tx=0x7f45a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2434f6b0918c48f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45ba7fc700 1 -- 172.21.15.32:0/4173687117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4011830 con 0x7f45bc15e160 2023-11-24T07:59:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45ba7fc700 1 -- 172.21.15.32:0/4173687117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4011990 con 0x7f45bc15e160 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.529+0000 7f45ba7fc700 1 -- 172.21.15.32:0/4173687117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4011d20 con 0x7f45bc15e160 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 -- 172.21.15.32:0/4173687117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc15e160 msgr2=0x7f45bc0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 --2- 172.21.15.32:0/4173687117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc15e160 0x7f45bc0b5da0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098f0 tx=0x7f45a4009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 -- 172.21.15.32:0/4173687117 shutdown_connections 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 --2- 172.21.15.32:0/4173687117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc15e160 0x7f45bc0b5da0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 --2- 172.21.15.32:0/4173687117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc0afc90 0x7f45bc073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 -- 172.21.15.32:0/4173687117 >> 172.21.15.32:0/4173687117 conn(0x7f45bc0d0bf0 msgr2=0x7f45bc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 -- 172.21.15.32:0/4173687117 shutdown_connections 2023-11-24T07:59:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.530+0000 7f45c1d61700 1 -- 172.21.15.32:0/4173687117 wait complete. 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.531+0000 7f45c1d61700 1 Processor -- start 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.531+0000 7f45c1d61700 1 -- start start 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45c1d61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45bc0744d0 0x7f45bc0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45c1d61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 0x7f45bc1f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45c1d61700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc15e160 0x7f45bc1f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45c1d61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc14bed0 con 0x7f45bc0afc90 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45c1d61700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c5c70 con 0x7f45bc0744d0 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45c1d61700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc1f38b0 con 0x7f45bc15e160 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 0x7f45bc1f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 0x7f45bc1f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58648/0 (socket says 172.21.15.32:58648) 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 -- 172.21.15.32:0/3778761742 learned_addr learned my addr 172.21.15.32:0/3778761742 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45bb7fe700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45bc0744d0 0x7f45bc0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45bbfff700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc15e160 0x7f45bc1f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 -- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc15e160 msgr2=0x7f45bc1f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc15e160 0x7f45bc1f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 -- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45bc0744d0 msgr2=0x7f45bc0b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45bc0744d0 0x7f45bc0b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 -- 172.21.15.32:0/3778761742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4009550 con 0x7f45bc0afc90 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.532+0000 7f45baffd700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 0x7f45bc1f2e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9e0 tx=0x7f45ac00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.533+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac009710 con 0x7f45bc0afc90 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.533+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac00ee80 con 0x7f45bc0afc90 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.533+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc0c5f40 con 0x7f45bc0afc90 2023-11-24T07:59:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.533+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00ca50 con 0x7f45bc0afc90 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.533+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc161070 con 0x7f45bc0afc90 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.534+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc0c6810 con 0x7f45bc0afc90 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.534+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45ac00cbb0 con 0x7f45bc0afc90 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.534+0000 7f45b3fff700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f459c06aed0 0x7f459c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.535+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45ac0895e0 con 0x7f45bc0afc90 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.535+0000 7f45bb7fe700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f459c06aed0 0x7f459c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.535+0000 7f45bb7fe700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f459c06aed0 0x7f459c06d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f45a4000c00 tx=0x7f45a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.539+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45ac054690 con 0x7f45bc0afc90 2023-11-24T07:59:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.743+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45bc0b51c0 con 0x7f45bc0afc90 2023-11-24T07:59:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.743+0000 7f45b3fff700 1 -- 172.21.15.32:0/3778761742 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45ac057cb0 con 0x7f45bc0afc90 2023-11-24T07:59:45.745 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f459c06aed0 msgr2=0x7f459c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f459c06aed0 0x7f459c06d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f45a4000c00 tx=0x7f45a4008030 comp rx=0 tx=0).stop 2023-11-24T07:59:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 msgr2=0x7f45bc1f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 0x7f45bc1f2e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9e0 tx=0x7f45ac00ecf0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 shutdown_connections 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f459c06aed0 0x7f459c06d390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45bc15e160 0x7f45bc1f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45bc0744d0 0x7f45bc0b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 --2- 172.21.15.32:0/3778761742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0afc90 0x7f45bc1f2e30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.745+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 >> 172.21.15.32:0/3778761742 conn(0x7f45bc0d0bf0 msgr2=0x7f45bc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.746+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 shutdown_connections 2023-11-24T07:59:45.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.746+0000 7f45c1d61700 1 -- 172.21.15.32:0/3778761742 wait complete. 2023-11-24T07:59:45.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.886+0000 7fb78207c700 1 Processor -- start 2023-11-24T07:59:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.886+0000 7fb78207c700 1 -- start start 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb78207c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 0x7fb77c0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb78207c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77c15f940 con 0x7fb77c144bb0 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb78207c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77c1604c0 con 0x7fb77c06c710 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb78207c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c15fb90 con 0x7fb77c0d0270 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb77c144bb0 0x7fb77c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60366/0 (socket says 172.21.15.32:60366) 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77b7fe700 1 -- 172.21.15.32:0/1737556666 learned_addr learned my addr 172.21.15.32:0/1737556666 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156692201 0 0) 0x7fb77c15f940 con 0x7fb77c144bb0 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7680035b0 con 0x7fb77c144bb0 2023-11-24T07:59:45.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77bfff700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 0x7fb77c0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436469440 0 0) 0x7fb7680035b0 con 0x7fb77c144bb0 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 msgr2=0x7fb77c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 0x7fb77c0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 >> v1:172.21.15.123:6789/0 conn(0x7fb77c06c710 legacy=0x7fb77c0b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c15fe70 con 0x7fb77c144bb0 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.887+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fb76c001670 con 0x7fb77c144bb0 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 >> v1:172.21.15.32:6789/0 conn(0x7fb77c144bb0 legacy=0x7fb77c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768003a30 0x7fb768003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 0x7fb768004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb768004d90 0x7fb768005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c15fb90 con 0x7fb768003a30 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76800bac0 con 0x7fb768004d90 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76800bc80 con 0x7fb77c0d0270 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768003a30 0x7fb768003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77affd700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 0x7fb768004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77bfff700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb768004d90 0x7fb768005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 -- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 msgr2=0x7fb768004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb77c0d0270 0x7fb768004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 -- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb768004d90 msgr2=0x7fb768005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb768004d90 0x7fb768005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 -- 172.21.15.32:0/1737556666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7680035b0 con 0x7fb768003a30 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768003a30 0x7fb768003e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007d30 tx=0x7fb768005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29557954b003a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00cd80 con 0x7fb768003a30 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c005960 con 0x7fb768003a30 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.888+0000 7fb77a7fc700 1 -- 172.21.15.32:0/1737556666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c005d10 con 0x7fb768003a30 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb78207c700 1 -- 172.21.15.32:0/1737556666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768003a30 msgr2=0x7fb768003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb78207c700 1 --2- 172.21.15.32:0/1737556666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb768003a30 0x7fb768003e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007d30 tx=0x7fb768005880 comp rx=0 tx=0).stop 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb77bfff700 1 -- 172.21.15.32:0/1737556666 reap_dead start 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb78207c700 1 -- 172.21.15.32:0/1737556666 shutdown_connections 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb78207c700 1 -- 172.21.15.32:0/1737556666 >> 172.21.15.32:0/1737556666 conn(0x7fb77c071a70 msgr2=0x7fb77c071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb78207c700 1 -- 172.21.15.32:0/1737556666 shutdown_connections 2023-11-24T07:59:45.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.889+0000 7fb78207c700 1 -- 172.21.15.32:0/1737556666 wait complete. 2023-11-24T07:59:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.890+0000 7fb78207c700 1 Processor -- start 2023-11-24T07:59:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.890+0000 7fb78207c700 1 -- start start 2023-11-24T07:59:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb78207c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768003a30 0x7fb77c143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb78207c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77c13fbe0 0x7fb77c13a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb78207c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 0x7fb77c140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb78207c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c13f750 con 0x7fb77c13e580 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb78207c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1441c0 con 0x7fb77c13fbe0 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb78207c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c13eb40 con 0x7fb768003a30 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb77bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 0x7fb77c140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb77bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 0x7fb77c140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58670/0 (socket says 172.21.15.32:58670) 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb77bfff700 1 -- 172.21.15.32:0/1559899369 learned_addr learned my addr 172.21.15.32:0/1559899369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb77affd700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77c13fbe0 0x7fb77c13a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.891+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768003a30 0x7fb77c143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77bfff700 1 -- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768003a30 msgr2=0x7fb77c143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77bfff700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768003a30 0x7fb77c143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77bfff700 1 -- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77c13fbe0 msgr2=0x7fb77c13a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77bfff700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77c13fbe0 0x7fb77c13a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77bfff700 1 -- 172.21.15.32:0/1559899369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c13ee20 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77affd700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77c13fbe0 0x7fb77c13a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768003a30 0x7fb77c143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb77bfff700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 0x7fb77c140af0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0cceb0 tx=0x7fb774011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400e040 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77400f040 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400fdb0 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c13f100 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.892+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c13a6b0 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.893+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77c068fb0 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.893+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb774022460 con 0x7fb77c13e580 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.894+0000 7fb773fff700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb75c06aed0 0x7fb75c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.894+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb774089b00 con 0x7fb77c13e580 2023-11-24T07:59:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.894+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb75c06aed0 0x7fb75c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.894+0000 7fb77b7fe700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb75c06aed0 0x7fb75c06d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb76c018000 tx=0x7fb76c00a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:45.898+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb774054bb0 con 0x7fb77c13e580 2023-11-24T07:59:46.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.099+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb77c0ccc90 con 0x7fb77c13e580 2023-11-24T07:59:46.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.099+0000 7fb773fff700 1 -- 172.21.15.32:0/1559899369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7740581d0 con 0x7fb77c13e580 2023-11-24T07:59:46.101 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb75c06aed0 msgr2=0x7fb75c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb75c06aed0 0x7fb75c06d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb76c018000 tx=0x7fb76c00a000 comp rx=0 tx=0).stop 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 msgr2=0x7fb77c140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 0x7fb77c140af0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0cceb0 tx=0x7fb774011f80 comp rx=0 tx=0).stop 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 shutdown_connections 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb75c06aed0 0x7fb75c06d390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb77c13e580 0x7fb77c140af0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb768003a30 0x7fb77c143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 --2- 172.21.15.32:0/1559899369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb77c13fbe0 0x7fb77c13a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 >> 172.21.15.32:0/1559899369 conn(0x7fb77c071a70 msgr2=0x7fb77c0b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 shutdown_connections 2023-11-24T07:59:46.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.102+0000 7fb78207c700 1 -- 172.21.15.32:0/1559899369 wait complete. 2023-11-24T07:59:46.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.245+0000 7fbc2af99700 1 Processor -- start 2023-11-24T07:59:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.245+0000 7fbc2af99700 1 -- start start 2023-11-24T07:59:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc2af99700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc2415e080 0x7fbc240cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc2af99700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240d0150 con 0x7fbc240ce4d0 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc2af99700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240cee50 con 0x7fbc240d22f0 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc2af99700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2415e460 con 0x7fbc2415e080 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc28d35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc2415e080 0x7fbc240cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc28d35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc2415e080 0x7fbc240cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58674/0 (socket says 172.21.15.32:58674) 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.246+0000 7fbc28d35700 1 -- 172.21.15.32:0/2202857200 learned_addr learned my addr 172.21.15.32:0/2202857200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc28d35700 1 -- 172.21.15.32:0/2202857200 >> v1:172.21.15.123:6789/0 conn(0x7fbc240d22f0 legacy=0x7fbc240d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc28d35700 1 -- 172.21.15.32:0/2202857200 >> v1:172.21.15.32:6789/0 conn(0x7fbc240ce4d0 legacy=0x7fbc240d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc28d35700 1 -- 172.21.15.32:0/2202857200 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240cf130 con 0x7fbc2415e080 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc237fe700 1 -- 172.21.15.32:0/2202857200 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335947968 0 0) 0x7fbc240d0150 con 0x7fbc240ce4d0 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc28d35700 1 --2- 172.21.15.32:0/2202857200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2415e080 0x7fbc240cc2a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098f0 tx=0x7fbc18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc6d090f8353c5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc237fe700 1 -- 172.21.15.32:0/2202857200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800b8a0 con 0x7fbc2415e080 2023-11-24T07:59:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc237fe700 1 -- 172.21.15.32:0/2202857200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1800ba00 con 0x7fbc2415e080 2023-11-24T07:59:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.247+0000 7fbc237fe700 1 -- 172.21.15.32:0/2202857200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800bd90 con 0x7fbc2415e080 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 -- 172.21.15.32:0/2202857200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2415e080 msgr2=0x7fbc240cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 --2- 172.21.15.32:0/2202857200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2415e080 0x7fbc240cc2a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098f0 tx=0x7fbc18009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 -- 172.21.15.32:0/2202857200 shutdown_connections 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 --2- 172.21.15.32:0/2202857200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc2415e080 0x7fbc240cc2a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 -- 172.21.15.32:0/2202857200 >> 172.21.15.32:0/2202857200 conn(0x7fbc24073130 msgr2=0x7fbc2409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 -- 172.21.15.32:0/2202857200 shutdown_connections 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.248+0000 7fbc2af99700 1 -- 172.21.15.32:0/2202857200 wait complete. 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.249+0000 7fbc2af99700 1 Processor -- start 2023-11-24T07:59:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.249+0000 7fbc2af99700 1 -- start start 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc2af99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 0x7fbc24144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc2af99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc240d22f0 0x7fbc240c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc2af99700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc2415e080 0x7fbc240c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc2af99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c96b0 con 0x7fbc240ce4d0 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc2af99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c5b40 con 0x7fbc240d22f0 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc2af99700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c9060 con 0x7fbc2415e080 2023-11-24T07:59:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 0x7fbc24144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 0x7fbc24144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58680/0 (socket says 172.21.15.32:58680) 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 -- 172.21.15.32:0/573485113 learned_addr learned my addr 172.21.15.32:0/573485113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 -- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc2415e080 msgr2=0x7fbc240c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc2415e080 0x7fbc240c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 -- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc240d22f0 msgr2=0x7fbc240c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc240d22f0 0x7fbc240c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 -- 172.21.15.32:0/573485113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18009550 con 0x7fbc240ce4d0 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.250+0000 7fbc28d35700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 0x7fbc24144ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc18014040 tx=0x7fbc18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.251+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011a70 con 0x7fbc240ce4d0 2023-11-24T07:59:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.251+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1801e070 con 0x7fbc240ce4d0 2023-11-24T07:59:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.251+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc240c9340 con 0x7fbc240ce4d0 2023-11-24T07:59:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.251+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011dd0 con 0x7fbc240ce4d0 2023-11-24T07:59:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.251+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc240c8150 con 0x7fbc240ce4d0 2023-11-24T07:59:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.252+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc24067f60 con 0x7fbc240ce4d0 2023-11-24T07:59:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.252+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc1802d450 con 0x7fbc240ce4d0 2023-11-24T07:59:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.252+0000 7fbc217fa700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbc0806aed0 0x7fbc0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.253+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc1808a6d0 con 0x7fbc240ce4d0 2023-11-24T07:59:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.253+0000 7fbc23fff700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbc0806aed0 0x7fbc0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.253+0000 7fbc23fff700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbc0806aed0 0x7fbc0806d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc10007fd0 tx=0x7fbc10007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.257+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc18055810 con 0x7fbc240ce4d0 2023-11-24T07:59:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.474+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbc240d19c0 con 0x7fbc240ce4d0 2023-11-24T07:59:46.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.475+0000 7fbc217fa700 1 -- 172.21.15.32:0/573485113 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbc18058e30 con 0x7fbc240ce4d0 2023-11-24T07:59:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbc0806aed0 msgr2=0x7fbc0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbc0806aed0 0x7fbc0806d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc10007fd0 tx=0x7fbc10007980 comp rx=0 tx=0).stop 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 msgr2=0x7fbc24144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 0x7fbc24144ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc18014040 tx=0x7fbc18004750 comp rx=0 tx=0).stop 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 shutdown_connections 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fbc0806aed0 0x7fbc0806d390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc2415e080 0x7fbc240c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc240ce4d0 0x7fbc24144ed0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 --2- 172.21.15.32:0/573485113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc240d22f0 0x7fbc240c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 >> 172.21.15.32:0/573485113 conn(0x7fbc24073130 msgr2=0x7fbc241609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 shutdown_connections 2023-11-24T07:59:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.477+0000 7fbc2af99700 1 -- 172.21.15.32:0/573485113 wait complete. 2023-11-24T07:59:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-11-24T07:59:46.491 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:59:46.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:46.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:46.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T07:59:46.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:46.530 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:59:46.587 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:46.587 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:46.609 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:46.609 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:46.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:46.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.795+0000 7fd06bfaf700 1 Processor -- start 2023-11-24T07:59:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.795+0000 7fd06bfaf700 1 -- start start 2023-11-24T07:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06bfaf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd0640adf00 0x7fd06406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06bfaf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 0x7fd0640b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06bfaf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0640af950 con 0x7fd0641446a0 2023-11-24T07:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06bfaf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064144a80 con 0x7fd0640adf00 2023-11-24T07:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06bfaf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06406c9c0 con 0x7fd06406c540 2023-11-24T07:59:46.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd069d4b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0641446a0 0x7fd0640b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60400/0 (socket says 172.21.15.32:60400) 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd069d4b700 1 -- 172.21.15.32:0/3570006420 learned_addr learned my addr 172.21.15.32:0/3570006420 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025211760 0 0) 0x7fd0640af950 con 0x7fd0641446a0 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06954a700 1 --2- 172.21.15.32:0/3570006420 >> v2:172.21.15.123:3300/0 conn(0x7fd0640adf00 0x7fd06406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd06a54c700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 0x7fd0640b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.796+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0500035b0 con 0x7fd0641446a0 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2998937163 0 0) 0x7fd0500035b0 con 0x7fd0641446a0 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 msgr2=0x7fd0640b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 0x7fd0640b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 >> v2:172.21.15.123:3300/0 conn(0x7fd0640adf00 msgr2=0x7fd06406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 --2- 172.21.15.32:0/3570006420 >> v2:172.21.15.123:3300/0 conn(0x7fd0640adf00 0x7fd06406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640afc30 con 0x7fd0641446a0 2023-11-24T07:59:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fd060001670 con 0x7fd0641446a0 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 >> v1:172.21.15.32:6789/0 conn(0x7fd0641446a0 legacy=0x7fd0640b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd050003a00 0x7fd050003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 0x7fd050004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd050005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064144a80 con 0x7fd050004d90 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06406c9c0 con 0x7fd050003a00 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05000ba10 con 0x7fd06406c540 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd050005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06954a700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 0x7fd050004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd069d4b700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd050003a00 0x7fd050003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 -- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 msgr2=0x7fd050004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd06406c540 0x7fd050004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 -- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd050003a00 msgr2=0x7fd050003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd050003a00 0x7fd050003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 -- 172.21.15.32:0/3570006420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0500035b0 con 0x7fd050004d90 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd06a54c700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd050005210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fc0 tx=0x7fd050005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c6ee22b55d7ca8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00ec30 con 0x7fd050004d90 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c00f040 con 0x7fd050004d90 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.797+0000 7fd068d49700 1 -- 172.21.15.32:0/3570006420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c014480 con 0x7fd050004d90 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06bfaf700 1 -- 172.21.15.32:0/3570006420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 msgr2=0x7fd050005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/3570006420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd050005210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fc0 tx=0x7fd050005880 comp rx=0 tx=0).stop 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06a54c700 1 -- 172.21.15.32:0/3570006420 reap_dead start 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06bfaf700 1 -- 172.21.15.32:0/3570006420 shutdown_connections 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06bfaf700 1 -- 172.21.15.32:0/3570006420 >> 172.21.15.32:0/3570006420 conn(0x7fd064071b90 msgr2=0x7fd064071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06bfaf700 1 -- 172.21.15.32:0/3570006420 shutdown_connections 2023-11-24T07:59:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.798+0000 7fd06bfaf700 1 -- 172.21.15.32:0/3570006420 wait complete. 2023-11-24T07:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.799+0000 7fd06bfaf700 1 Processor -- start 2023-11-24T07:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.799+0000 7fd06bfaf700 1 -- start start 2023-11-24T07:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd06bfaf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd06415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd06bfaf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd064073f90 0x7fd064074410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd06bfaf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd064137d60 0x7fd064138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd06bfaf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064074950 con 0x7fd050004d90 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd06bfaf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06409bf50 con 0x7fd064137d60 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd06bfaf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd064138ad0 con 0x7fd064073f90 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd06415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd06415a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58700/0 (socket says 172.21.15.32:58700) 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 -- 172.21.15.32:0/1527347220 learned_addr learned my addr 172.21.15.32:0/1527347220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 -- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd064073f90 msgr2=0x7fd064074410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd064073f90 0x7fd064074410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 -- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd064137d60 msgr2=0x7fd064138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd064137d60 0x7fd064138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.800+0000 7fd069d4b700 1 -- 172.21.15.32:0/1527347220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064138db0 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.801+0000 7fd069d4b700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd06415a5c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd060000fb0 tx=0x7fd0600076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.801+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000f7c0 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.801+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06000f940 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.801+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd064139090 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.801+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000fcf0 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.801+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06415ab00 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.802+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06409c230 con 0x7fd050004d90 2023-11-24T07:59:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.802+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd06001d460 con 0x7fd050004d90 2023-11-24T07:59:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.802+0000 7fd05a7fc700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd04806ae60 0x7fd04806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.803+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0600895a0 con 0x7fd050004d90 2023-11-24T07:59:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.803+0000 7fd06954a700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd04806ae60 0x7fd04806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.803+0000 7fd06954a700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd04806ae60 0x7fd04806d320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd054006b20 tx=0x7fd054006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:46.807+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0600546e0 con 0x7fd050004d90 2023-11-24T07:59:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.013+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0640753f0 con 0x7fd050004d90 2023-11-24T07:59:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.014+0000 7fd05a7fc700 1 -- 172.21.15.32:0/1527347220 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fd060057d00 con 0x7fd050004d90 2023-11-24T07:59:47.016 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:47.016 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:35.687434+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:47.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd04806ae60 msgr2=0x7fd04806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd04806ae60 0x7fd04806d320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd054006b20 tx=0x7fd054006ab0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 msgr2=0x7fd06415a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd06415a5c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd060000fb0 tx=0x7fd0600076a0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 shutdown_connections 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd04806ae60 0x7fd04806d320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd064137d60 0x7fd064138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050004d90 0x7fd06415a5c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 --2- 172.21.15.32:0/1527347220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd064073f90 0x7fd064074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.016+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 >> 172.21.15.32:0/1527347220 conn(0x7fd064071b90 msgr2=0x7fd064143430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.017+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 shutdown_connections 2023-11-24T07:59:47.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.017+0000 7fd06bfaf700 1 -- 172.21.15.32:0/1527347220 wait complete. 2023-11-24T07:59:47.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:32.221808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:33.067627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:47.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.157+0000 7f8aaec80700 1 Processor -- start 2023-11-24T07:59:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.157+0000 7f8aaec80700 1 -- start start 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaec80700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaec80700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8aa80ce830 0x7f8aa80cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaec80700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa80cf1f0 0x7f8aa80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaec80700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa815c330 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaec80700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa80d0140 con 0x7f8aa80ce830 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaec80700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80d03e0 con 0x7f8aa815bf50 2023-11-24T07:59:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa80cf1f0 0x7f8aa80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa80cf1f0 0x7f8aa80c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58712/0 (socket says 172.21.15.32:58712) 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 -- 172.21.15.32:0/4105133413 learned_addr learned my addr 172.21.15.32:0/4105133413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aa7fff700 1 --2- 172.21.15.32:0/4105133413 >> v2:172.21.15.123:3300/0 conn(0x7f8aa80ce830 0x7f8aa80cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aaca1c700 1 --2- 172.21.15.32:0/4105133413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 -- 172.21.15.32:0/4105133413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 msgr2=0x7f8aa80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 --2- 172.21.15.32:0/4105133413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 -- 172.21.15.32:0/4105133413 >> v2:172.21.15.123:3300/0 conn(0x7f8aa80ce830 msgr2=0x7f8aa80cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 --2- 172.21.15.32:0/4105133413 >> v2:172.21.15.123:3300/0 conn(0x7f8aa80ce830 0x7f8aa80cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 -- 172.21.15.32:0/4105133413 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80d06c0 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aa7fff700 1 --2- 172.21.15.32:0/4105133413 >> v2:172.21.15.123:3300/0 conn(0x7f8aa80ce830 0x7f8aa80cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.158+0000 7f8aad21d700 1 --2- 172.21.15.32:0/4105133413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80c5c20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8071c80 tx=0x7f8aa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5958354176c14bad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aa77fe700 1 -- 172.21.15.32:0/4105133413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000e070 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aa77fe700 1 -- 172.21.15.32:0/4105133413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa000b9c0 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aa77fe700 1 -- 172.21.15.32:0/4105133413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000bd70 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 -- 172.21.15.32:0/4105133413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 msgr2=0x7f8aa80c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 --2- 172.21.15.32:0/4105133413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80c5c20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8071c80 tx=0x7f8aa000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 -- 172.21.15.32:0/4105133413 shutdown_connections 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 --2- 172.21.15.32:0/4105133413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80c5c20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 --2- 172.21.15.32:0/4105133413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 --2- 172.21.15.32:0/4105133413 >> v2:172.21.15.123:3300/0 conn(0x7f8aa80ce830 0x7f8aa80cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 -- 172.21.15.32:0/4105133413 >> 172.21.15.32:0/4105133413 conn(0x7f8aa80b3780 msgr2=0x7f8aa80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 -- 172.21.15.32:0/4105133413 shutdown_connections 2023-11-24T07:59:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.159+0000 7f8aaec80700 1 -- 172.21.15.32:0/4105133413 wait complete. 2023-11-24T07:59:47.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.160+0000 7f8aaec80700 1 Processor -- start 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.160+0000 7f8aaec80700 1 -- start start 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aaec80700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa80ce830 0x7f8aa8159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aaec80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aaec80700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aaec80700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80b4aa0 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aaec80700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa806baa0 con 0x7f8aa80ce830 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aaec80700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa806bd80 con 0x7f8aa815bf50 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.161+0000 7f8aa7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58716/0 (socket says 172.21.15.32:58716) 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 -- 172.21.15.32:0/1968481097 learned_addr learned my addr 172.21.15.32:0/1968481097 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aad21d700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aaca1c700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa80ce830 0x7f8aa8159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 -- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 msgr2=0x7f8aa80b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 -- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa80ce830 msgr2=0x7f8aa8159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa80ce830 0x7f8aa8159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 -- 172.21.15.32:0/1968481097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa0009b30 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aaca1c700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa80ce830 0x7f8aa8159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aad21d700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa7fff700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80b6950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c750 tx=0x7f8a9400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011800 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94011960 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011cd0 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa806c060 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.162+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa806c620 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.163+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa806c7e0 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.164+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a94021350 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.164+0000 7f8aa57fa700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8a8c06aed0 0x7f8a8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.164+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a94089570 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.164+0000 7f8aaca1c700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8a8c06aed0 0x7f8a8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.165+0000 7f8aaca1c700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8a8c06aed0 0x7f8a8c06d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006fd0 tx=0x7f8a9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.168+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a94054620 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.368+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8aa8068fb0 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.369+0000 7f8aa57fa700 1 -- 172.21.15.32:0/1968481097 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a9405e020 con 0x7f8aa80cf1f0 2023-11-24T07:59:47.371 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8a8c06aed0 msgr2=0x7f8a8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8a8c06aed0 0x7f8a8c06d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006fd0 tx=0x7f8a9c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 msgr2=0x7f8aa80b6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80b6950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c750 tx=0x7f8a9400cb10 comp rx=0 tx=0).stop 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 shutdown_connections 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8a8c06aed0 0x7f8a8c06d390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8aa815bf50 0x7f8aa80b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa80ce830 0x7f8aa8159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 --2- 172.21.15.32:0/1968481097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa80cf1f0 0x7f8aa80b6950 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 >> 172.21.15.32:0/1968481097 conn(0x7f8aa80b3780 msgr2=0x7f8aa80c7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 shutdown_connections 2023-11-24T07:59:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.371+0000 7f8aaec80700 1 -- 172.21.15.32:0/1968481097 wait complete. 2023-11-24T07:59:47.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.510+0000 7fb11a687700 1 Processor -- start 2023-11-24T07:59:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.510+0000 7fb11a687700 1 -- start start 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb11a687700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1141587f0 0x7fb11406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb11a687700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 0x7fb11415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb11a687700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb1140b3c60 0x7fb1140b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb11a687700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1140b4610 con 0x7fb1141587f0 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb11a687700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11406b530 con 0x7fb1140b3c60 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb11a687700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1140ae5e0 con 0x7fb11415b370 2023-11-24T07:59:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb113fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1141587f0 0x7fb11406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb113fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1141587f0 0x7fb11406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58722/0 (socket says 172.21.15.32:58722) 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.511+0000 7fb113fff700 1 -- 172.21.15.32:0/251812613 learned_addr learned my addr 172.21.15.32:0/251812613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb118c24700 1 --2- 172.21.15.32:0/251812613 >> v2:172.21.15.123:3300/0 conn(0x7fb1140b3c60 0x7fb1140b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb113fff700 1 -- 172.21.15.32:0/251812613 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 msgr2=0x7fb11415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb113fff700 1 --2- 172.21.15.32:0/251812613 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 0x7fb11415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb113fff700 1 -- 172.21.15.32:0/251812613 >> v2:172.21.15.123:3300/0 conn(0x7fb1140b3c60 msgr2=0x7fb1140b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb113fff700 1 --2- 172.21.15.32:0/251812613 >> v2:172.21.15.123:3300/0 conn(0x7fb1140b3c60 0x7fb1140b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb113fff700 1 -- 172.21.15.32:0/251812613 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11415be30 con 0x7fb1141587f0 2023-11-24T07:59:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb113fff700 1 --2- 172.21.15.32:0/251812613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1141587f0 0x7fb11406aff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb11415bd10 tx=0x7fb104009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45ae44fa51e7a43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb112ffd700 1 -- 172.21.15.32:0/251812613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400b870 con 0x7fb1141587f0 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb112ffd700 1 -- 172.21.15.32:0/251812613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10400b9d0 con 0x7fb1141587f0 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.512+0000 7fb112ffd700 1 -- 172.21.15.32:0/251812613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400bd60 con 0x7fb1141587f0 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 -- 172.21.15.32:0/251812613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1141587f0 msgr2=0x7fb11406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 --2- 172.21.15.32:0/251812613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1141587f0 0x7fb11406aff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb11415bd10 tx=0x7fb104009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 -- 172.21.15.32:0/251812613 shutdown_connections 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 --2- 172.21.15.32:0/251812613 >> v2:172.21.15.123:3300/0 conn(0x7fb1140b3c60 0x7fb1140b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 --2- 172.21.15.32:0/251812613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1141587f0 0x7fb11406aff0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 --2- 172.21.15.32:0/251812613 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 0x7fb11415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 -- 172.21.15.32:0/251812613 >> 172.21.15.32:0/251812613 conn(0x7fb1140d0bf0 msgr2=0x7fb1140d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 -- 172.21.15.32:0/251812613 shutdown_connections 2023-11-24T07:59:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.513+0000 7fb11a687700 1 -- 172.21.15.32:0/251812613 wait complete. 2023-11-24T07:59:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.514+0000 7fb11a687700 1 Processor -- start 2023-11-24T07:59:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.514+0000 7fb11a687700 1 -- start start 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb11a687700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 0x7fb1141519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb11a687700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1141587f0 0x7fb11415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb11a687700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 0x7fb11415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb11a687700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114152460 con 0x7fb1140b3c60 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb11a687700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11415a5e0 con 0x7fb1141587f0 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb11a687700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11415a020 con 0x7fb11415b370 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 0x7fb1141519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 0x7fb1141519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58728/0 (socket says 172.21.15.32:58728) 2023-11-24T07:59:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 -- 172.21.15.32:0/1535039286 learned_addr learned my addr 172.21.15.32:0/1535039286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb1137fe700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1141587f0 0x7fb11415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 -- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 msgr2=0x7fb11415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 0x7fb11415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 -- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1141587f0 msgr2=0x7fb11415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1141587f0 0x7fb11415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 -- 172.21.15.32:0/1535039286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009550 con 0x7fb1140b3c60 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb113fff700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 0x7fb1141519a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb1140d1730 tx=0x7fb104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104004310 con 0x7fb1140b3c60 2023-11-24T07:59:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.515+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10401f070 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.516+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11415a300 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.516+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1040036a0 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.516+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb114159330 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.517+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb114155300 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.517+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb104021030 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.517+0000 7fb110ff9700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb10006aed0 0x7fb10006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.518+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb10408ab00 con 0x7fb1140b3c60 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.518+0000 7fb1137fe700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb10006aed0 0x7fb10006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.518+0000 7fb1137fe700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb10006aed0 0x7fb10006d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb10800a810 tx=0x7fb108008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.522+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb10408ce60 con 0x7fb1140b3c60 2023-11-24T07:59:47.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.721+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1141525a0 con 0x7fb1140b3c60 2023-11-24T07:59:47.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.722+0000 7fb110ff9700 1 -- 172.21.15.32:0/1535039286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1141525a0 con 0x7fb1140b3c60 2023-11-24T07:59:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:47.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb10006aed0 msgr2=0x7fb10006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb10006aed0 0x7fb10006d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb10800a810 tx=0x7fb108008040 comp rx=0 tx=0).stop 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 msgr2=0x7fb1141519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 0x7fb1141519a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb1140d1730 tx=0x7fb104004750 comp rx=0 tx=0).stop 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 shutdown_connections 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb10006aed0 0x7fb10006d390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb11415b370 0x7fb11415c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1140b3c60 0x7fb1141519a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 --2- 172.21.15.32:0/1535039286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1141587f0 0x7fb11415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 >> 172.21.15.32:0/1535039286 conn(0x7fb1140d0bf0 msgr2=0x7fb11406e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 shutdown_connections 2023-11-24T07:59:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.724+0000 7fb11a687700 1 -- 172.21.15.32:0/1535039286 wait complete. 2023-11-24T07:59:47.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:47.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.864+0000 7fcd97893700 1 Processor -- start 2023-11-24T07:59:47.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd97893700 1 -- start start 2023-11-24T07:59:47.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd97893700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd901587f0 0x7fcd9006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd97893700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9015b370 0x7fcd9015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd97893700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9006b530 con 0x7fcd900b3c60 2023-11-24T07:59:47.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd97893700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd900ae5e0 con 0x7fcd901587f0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd97893700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9015be30 con 0x7fcd9015b370 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd9562f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd901587f0 0x7fcd9006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd9562f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd901587f0 0x7fcd9006aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58736/0 (socket says 172.21.15.32:58736) 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.865+0000 7fcd9562f700 1 -- 172.21.15.32:0/3355429150 learned_addr learned my addr 172.21.15.32:0/3355429150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd94e2e700 1 --2- 172.21.15.32:0/3355429150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9015b370 0x7fcd9015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd9562f700 1 -- 172.21.15.32:0/3355429150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9015b370 msgr2=0x7fcd9015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd9562f700 1 --2- 172.21.15.32:0/3355429150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9015b370 0x7fcd9015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd9562f700 1 -- 172.21.15.32:0/3355429150 >> v1:172.21.15.123:6789/0 conn(0x7fcd900b3c60 legacy=0x7fcd900af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd9562f700 1 -- 172.21.15.32:0/3355429150 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9015c110 con 0x7fcd901587f0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd9562f700 1 --2- 172.21.15.32:0/3355429150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd901587f0 0x7fcd9006aff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcd800098f0 tx=0x7fcd80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f68bf14ae27b99 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd87fff700 1 -- 172.21.15.32:0/3355429150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000b8a0 con 0x7fcd901587f0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd87fff700 1 -- 172.21.15.32:0/3355429150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8000ba00 con 0x7fcd901587f0 2023-11-24T07:59:47.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.866+0000 7fcd87fff700 1 -- 172.21.15.32:0/3355429150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000bd90 con 0x7fcd901587f0 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 -- 172.21.15.32:0/3355429150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd901587f0 msgr2=0x7fcd9006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 --2- 172.21.15.32:0/3355429150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd901587f0 0x7fcd9006aff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcd800098f0 tx=0x7fcd80009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 -- 172.21.15.32:0/3355429150 shutdown_connections 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 --2- 172.21.15.32:0/3355429150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd901587f0 0x7fcd9006aff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 --2- 172.21.15.32:0/3355429150 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9015b370 0x7fcd9015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 -- 172.21.15.32:0/3355429150 >> 172.21.15.32:0/3355429150 conn(0x7fcd900d0bf0 msgr2=0x7fcd900d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 -- 172.21.15.32:0/3355429150 shutdown_connections 2023-11-24T07:59:47.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.867+0000 7fcd97893700 1 -- 172.21.15.32:0/3355429150 wait complete. 2023-11-24T07:59:47.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.868+0000 7fcd97893700 1 Processor -- start 2023-11-24T07:59:47.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.868+0000 7fcd97893700 1 -- start start 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd97893700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd900b3c60 0x7fcd901481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd97893700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd901587f0 0x7fcd900b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd97893700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 0x7fcd90150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd97893700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9014ae50 con 0x7fcd9015b370 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd97893700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90148af0 con 0x7fcd900b3c60 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd97893700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90150870 con 0x7fcd901587f0 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd95e30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 0x7fcd90150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.869+0000 7fcd95e30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 0x7fcd90150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58746/0 (socket says 172.21.15.32:58746) 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 -- 172.21.15.32:0/2543815334 learned_addr learned my addr 172.21.15.32:0/2543815334 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd9562f700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd900b3c60 0x7fcd901481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd94e2e700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd901587f0 0x7fcd900b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 -- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd901587f0 msgr2=0x7fcd900b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd901587f0 0x7fcd900b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 -- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd900b3c60 msgr2=0x7fcd901481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd900b3c60 0x7fcd901481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 -- 172.21.15.32:0/2543815334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd80009550 con 0x7fcd9015b370 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd9562f700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd900b3c60 0x7fcd901481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd95e30700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 0x7fcd90150330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800cc50 tx=0x7fcd8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800ad80 con 0x7fcd9015b370 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8800ce50 con 0x7fcd9015b370 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd90148810 con 0x7fcd9015b370 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88003c40 con 0x7fcd9015b370 2023-11-24T07:59:47.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.870+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9014b360 con 0x7fcd9015b370 2023-11-24T07:59:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.871+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd90068fb0 con 0x7fcd9015b370 2023-11-24T07:59:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.872+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd880050d0 con 0x7fcd9015b370 2023-11-24T07:59:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.872+0000 7fcd85ffb700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcd7406aed0 0x7fcd7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.872+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd88089750 con 0x7fcd9015b370 2023-11-24T07:59:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.873+0000 7fcd9562f700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcd7406aed0 0x7fcd7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.873+0000 7fcd9562f700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcd7406aed0 0x7fcd7406d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd800098f0 tx=0x7fcd80008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:47.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:47.877+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd88054760 con 0x7fcd9015b370 2023-11-24T07:59:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.076+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd9006beb0 con 0x7fcd9015b370 2023-11-24T07:59:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.076+0000 7fcd85ffb700 1 -- 172.21.15.32:0/2543815334 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd88057d80 con 0x7fcd9015b370 2023-11-24T07:59:48.078 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcd7406aed0 msgr2=0x7fcd7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcd7406aed0 0x7fcd7406d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd800098f0 tx=0x7fcd80008030 comp rx=0 tx=0).stop 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 msgr2=0x7fcd90150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 0x7fcd90150330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800cc50 tx=0x7fcd8800a340 comp rx=0 tx=0).stop 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 shutdown_connections 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fcd7406aed0 0x7fcd7406d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9015b370 0x7fcd90150330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd900b3c60 0x7fcd901481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 --2- 172.21.15.32:0/2543815334 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd901587f0 0x7fcd900b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 >> 172.21.15.32:0/2543815334 conn(0x7fcd900d0bf0 msgr2=0x7fcd90151000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 shutdown_connections 2023-11-24T07:59:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.079+0000 7fcd97893700 1 -- 172.21.15.32:0/2543815334 wait complete. 2023-11-24T07:59:48.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:48.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.217+0000 7f9cb7cfe700 1 Processor -- start 2023-11-24T07:59:48.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.217+0000 7f9cb7cfe700 1 -- start start 2023-11-24T07:59:48.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb7cfe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb7cfe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cb00afe70 0x7f9cb00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb7cfe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cb00b4b40 0x7f9cb01624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb7cfe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb006dd80 con 0x7f9cb00b4b40 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb7cfe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb015f880 con 0x7f9cb00afe70 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb7cfe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb015ed90 con 0x7f9cb015bef0 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cb00b4b40 0x7f9cb01624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cb00b4b40 0x7f9cb01624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58754/0 (socket says 172.21.15.32:58754) 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 -- 172.21.15.32:0/1781632086 learned_addr learned my addr 172.21.15.32:0/1781632086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb5a9a700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb5299700 1 --2- 172.21.15.32:0/1781632086 >> v2:172.21.15.123:3300/0 conn(0x7f9cb00afe70 0x7f9cb00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 -- 172.21.15.32:0/1781632086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 msgr2=0x7f9cb015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 -- 172.21.15.32:0/1781632086 >> v2:172.21.15.123:3300/0 conn(0x7f9cb00afe70 msgr2=0x7f9cb00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 --2- 172.21.15.32:0/1781632086 >> v2:172.21.15.123:3300/0 conn(0x7f9cb00afe70 0x7f9cb00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 -- 172.21.15.32:0/1781632086 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb015c2d0 con 0x7f9cb00b4b40 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb5299700 1 --2- 172.21.15.32:0/1781632086 >> v2:172.21.15.123:3300/0 conn(0x7f9cb00afe70 0x7f9cb00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb5a9a700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb629b700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb01624d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00716f0 tx=0x7f9ca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d06712defcda78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.218+0000 7f9cb4a98700 1 -- 172.21.15.32:0/1781632086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800e070 con 0x7f9cb00b4b40 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb4a98700 1 -- 172.21.15.32:0/1781632086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca800b9c0 con 0x7f9cb00b4b40 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb4a98700 1 -- 172.21.15.32:0/1781632086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800bd70 con 0x7f9cb00b4b40 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1781632086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 msgr2=0x7f9cb01624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb01624d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00716f0 tx=0x7f9ca800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1781632086 shutdown_connections 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb01624d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1781632086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1781632086 >> v2:172.21.15.123:3300/0 conn(0x7f9cb00afe70 0x7f9cb00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1781632086 >> 172.21.15.32:0/1781632086 conn(0x7f9cb0071ac0 msgr2=0x7f9cb0071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1781632086 shutdown_connections 2023-11-24T07:59:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.219+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1781632086 wait complete. 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.220+0000 7f9cb7cfe700 1 Processor -- start 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.220+0000 7f9cb7cfe700 1 -- start start 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb7cfe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb00afe70 0x7f9cb0152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb7cfe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb7cfe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb01573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb7cfe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb015c4b0 con 0x7f9cb00b4b40 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb7cfe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb01525b0 con 0x7f9cb00afe70 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb7cfe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb0154160 con 0x7f9cb015bef0 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb5299700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb5299700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb0156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58758/0 (socket says 172.21.15.32:58758) 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb5299700 1 -- 172.21.15.32:0/1855057590 learned_addr learned my addr 172.21.15.32:0/1855057590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb5a9a700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb00afe70 0x7f9cb0152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.221+0000 7f9cb629b700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb01573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb5299700 1 -- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 msgr2=0x7f9cb01573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb5299700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb01573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb5299700 1 -- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb00afe70 msgr2=0x7f9cb0152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb5299700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb00afe70 0x7f9cb0152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb5299700 1 -- 172.21.15.32:0/1855057590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8009b30 con 0x7f9cb00b4b40 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb629b700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb01573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb5299700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb0156e80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000d800 tx=0x7f9ca000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0004030 con 0x7f9cb00b4b40 2023-11-24T07:59:48.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0004190 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb0154440 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0004520 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.222+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb0154a00 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.223+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb006f910 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.224+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ca00036a0 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.224+0000 7f9ca67fc700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9c9406aed0 0x7f9c9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.224+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ca0088c90 con 0x7f9cb00b4b40 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.224+0000 7f9cb5a9a700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9c9406aed0 0x7f9c9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.225+0000 7f9cb5a9a700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9c9406aed0 0x7f9c9406d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c940 tx=0x7f9cac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.229+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ca0053d60 con 0x7f9cb00b4b40 2023-11-24T07:59:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.443+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9cb006dba0 con 0x7f9cb00b4b40 2023-11-24T07:59:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.444+0000 7f9ca67fc700 1 -- 172.21.15.32:0/1855057590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9ca0018070 con 0x7f9cb00b4b40 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9c9406aed0 msgr2=0x7f9c9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9c9406aed0 0x7f9c9406d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c940 tx=0x7f9cac008040 comp rx=0 tx=0).stop 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 msgr2=0x7f9cb0156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb0156e80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000d800 tx=0x7f9ca000db10 comp rx=0 tx=0).stop 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 shutdown_connections 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9c9406aed0 0x7f9c9406d390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9cb015bef0 0x7f9cb01573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb00afe70 0x7f9cb0152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.445+0000 7f9cb7cfe700 1 --2- 172.21.15.32:0/1855057590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00b4b40 0x7f9cb0156e80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.446+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 >> 172.21.15.32:0/1855057590 conn(0x7f9cb0071ac0 msgr2=0x7f9cb0155740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.446+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 shutdown_connections 2023-11-24T07:59:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.446+0000 7f9cb7cfe700 1 -- 172.21.15.32:0/1855057590 wait complete. 2023-11-24T07:59:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-11-24T07:59:48.459 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:59:48.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:48.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:48.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T07:59:48.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:48.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:59:48.531 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T07:59:48.531 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:48.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17559 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:59:48.549 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T07:59:48.549 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:48.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18655 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:59:48.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.716+0000 7f52ee00e700 1 Processor -- start 2023-11-24T07:59:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.716+0000 7f52ee00e700 1 -- start start 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.716+0000 7f52ee00e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52e80d1ee0 0x7f52e80b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.716+0000 7f52ee00e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e80b9e10 con 0x7f52e80b6e80 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.716+0000 7f52ee00e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e815e3e0 con 0x7f52e815e000 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.716+0000 7f52ee00e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80b8070 con 0x7f52e80d1ee0 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52e80d1ee0 0x7f52e80b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52e80d1ee0 0x7f52e80b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58766/0 (socket says 172.21.15.32:58766) 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 -- 172.21.15.32:0/1493758590 learned_addr learned my addr 172.21.15.32:0/1493758590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 -- 172.21.15.32:0/1493758590 >> v1:172.21.15.123:6789/0 conn(0x7f52e815e000 legacy=0x7f52e80b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 -- 172.21.15.32:0/1493758590 >> v1:172.21.15.32:6789/0 conn(0x7f52e80b6e80 legacy=0x7f52e8167b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 -- 172.21.15.32:0/1493758590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80b73f0 con 0x7f52e80d1ee0 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52deffd700 1 --2- 172.21.15.32:0/1493758590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e80b5940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f52d00098f0 tx=0x7f52d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f4b3209a6831c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52e6ffd700 1 -- 172.21.15.32:0/1493758590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000b8a0 con 0x7f52e80d1ee0 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52e6ffd700 1 -- 172.21.15.32:0/1493758590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d000ba00 con 0x7f52e80d1ee0 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.717+0000 7f52e6ffd700 1 -- 172.21.15.32:0/1493758590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000bd90 con 0x7f52e80d1ee0 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 -- 172.21.15.32:0/1493758590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 msgr2=0x7f52e80b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 --2- 172.21.15.32:0/1493758590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e80b5940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f52d00098f0 tx=0x7f52d0009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 -- 172.21.15.32:0/1493758590 shutdown_connections 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 --2- 172.21.15.32:0/1493758590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e80b5940 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 -- 172.21.15.32:0/1493758590 >> 172.21.15.32:0/1493758590 conn(0x7f52e806b410 msgr2=0x7f52e806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 -- 172.21.15.32:0/1493758590 shutdown_connections 2023-11-24T07:59:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.718+0000 7f52ee00e700 1 -- 172.21.15.32:0/1493758590 wait complete. 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.719+0000 7f52ee00e700 1 Processor -- start 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.719+0000 7f52ee00e700 1 -- start start 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52ee00e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e80b6e80 0x7f52e814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52ee00e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e8158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52ee00e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e815e000 0x7f52e8159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52ee00e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e80caf90 con 0x7f52e80d1ee0 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52ee00e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e80cb830 con 0x7f52e815e000 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52ee00e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81599d0 con 0x7f52e80b6e80 2023-11-24T07:59:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e8158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e8158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58772/0 (socket says 172.21.15.32:58772) 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 -- 172.21.15.32:0/836039615 learned_addr learned my addr 172.21.15.32:0/836039615 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52e7fff700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e815e000 0x7f52e8159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 -- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e80b6e80 msgr2=0x7f52e814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e80b6e80 0x7f52e814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 -- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e815e000 msgr2=0x7f52e8159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e815e000 0x7f52e8159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.720+0000 7f52deffd700 1 -- 172.21.15.32:0/836039615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0009550 con 0x7f52e80d1ee0 2023-11-24T07:59:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.721+0000 7f52deffd700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e8158f50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009c40 tx=0x7f52d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.721+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0004530 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.721+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d001f070 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.721+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8159c50 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.721+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00038a0 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.721+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e80cb4c0 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.722+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e80b9660 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.722+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52d0021030 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.723+0000 7f52e4ff9700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52c806aed0 0x7f52c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.723+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52d008aa70 con 0x7f52e80d1ee0 2023-11-24T07:59:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.723+0000 7f52e77fe700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52c806aed0 0x7f52c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.724+0000 7f52e77fe700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52c806aed0 0x7f52c806d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52d80015f0 tx=0x7f52d8007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.728+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52d0055cd0 con 0x7f52e80d1ee0 2023-11-24T07:59:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.979+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52e8067f60 con 0x7f52e80d1ee0 2023-11-24T07:59:48.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.980+0000 7f52e4ff9700 1 -- 172.21.15.32:0/836039615 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f52d0015090 con 0x7f52e80d1ee0 2023-11-24T07:59:48.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:48.982 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:34.106005+0000","modified":"2023-11-24T07:59:37.114690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/598023464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":598023464},{"type":"v1","addr":"172.21.15.32:6835","nonce":598023464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52c806aed0 msgr2=0x7f52c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52c806aed0 0x7f52c806d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52d80015f0 tx=0x7f52d8007a00 comp rx=0 tx=0).stop 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 msgr2=0x7f52e8158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e8158f50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009c40 tx=0x7f52d0004750 comp rx=0 tx=0).stop 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 shutdown_connections 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f52c806aed0 0x7f52c806d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e815e000 0x7f52e8159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e80b6e80 0x7f52e814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 --2- 172.21.15.32:0/836039615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e80d1ee0 0x7f52e8158f50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 >> 172.21.15.32:0/836039615 conn(0x7f52e806b410 msgr2=0x7f52e80b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 shutdown_connections 2023-11-24T07:59:48.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:48.982+0000 7f52ee00e700 1 -- 172.21.15.32:0/836039615 wait complete. 2023-11-24T07:59:48.987 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2023-11-24T07:59:48.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.119+0000 7f09d5d08700 1 Processor -- start 2023-11-24T07:59:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.119+0000 7f09d5d08700 1 -- start start 2023-11-24T07:59:49.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.120+0000 7f09d5d08700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09d015be00 0x7f09d00cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09d5d08700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d015c1e0 0x7f09d00aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09d5d08700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00b00e0 con 0x7f09d00af030 2023-11-24T07:59:49.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09d5d08700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00af4d0 con 0x7f09d015c1e0 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09d5d08700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00af7b0 con 0x7f09d015be00 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d015c1e0 0x7f09d00aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09d015c1e0 0x7f09d00aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58780/0 (socket says 172.21.15.32:58780) 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09cffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09d00af030 0x7f09d00d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60494/0 (socket says 172.21.15.32:60494) 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 -- 172.21.15.32:0/2339939242 learned_addr learned my addr 172.21.15.32:0/2339939242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/2339939242 >> v2:172.21.15.123:3300/0 conn(0x7f09d015be00 0x7f09d00cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 -- 172.21.15.32:0/2339939242 >> v2:172.21.15.123:3300/0 conn(0x7f09d015be00 msgr2=0x7f09d00cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 --2- 172.21.15.32:0/2339939242 >> v2:172.21.15.123:3300/0 conn(0x7f09d015be00 0x7f09d00cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 -- 172.21.15.32:0/2339939242 >> v1:172.21.15.32:6789/0 conn(0x7f09d00af030 legacy=0x7f09d00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 -- 172.21.15.32:0/2339939242 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00afa90 con 0x7f09d015c1e0 2023-11-24T07:59:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ceffd700 1 --2- 172.21.15.32:0/2339939242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015c1e0 0x7f09d00aeaf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f09d00d28d0 tx=0x7f09c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10b7b7c6d85ded1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ce7fc700 1 -- 172.21.15.32:0/2339939242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4011830 con 0x7f09d015c1e0 2023-11-24T07:59:49.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ce7fc700 1 -- 172.21.15.32:0/2339939242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c4011990 con 0x7f09d015c1e0 2023-11-24T07:59:49.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.121+0000 7f09ce7fc700 1 -- 172.21.15.32:0/2339939242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4011d20 con 0x7f09d015c1e0 2023-11-24T07:59:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 -- 172.21.15.32:0/2339939242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015c1e0 msgr2=0x7f09d00aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2339939242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015c1e0 0x7f09d00aeaf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f09d00d28d0 tx=0x7f09c4009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 -- 172.21.15.32:0/2339939242 shutdown_connections 2023-11-24T07:59:49.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2339939242 >> v2:172.21.15.123:3300/0 conn(0x7f09d015be00 0x7f09d00cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2339939242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015c1e0 0x7f09d00aeaf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 -- 172.21.15.32:0/2339939242 >> 172.21.15.32:0/2339939242 conn(0x7f09d009aa00 msgr2=0x7f09d009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 -- 172.21.15.32:0/2339939242 shutdown_connections 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.122+0000 7f09d5d08700 1 -- 172.21.15.32:0/2339939242 wait complete. 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.123+0000 7f09d5d08700 1 Processor -- start 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.123+0000 7f09d5d08700 1 -- start start 2023-11-24T07:59:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09d5d08700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d00af030 0x7f09d015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09d5d08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 0x7f09d00c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09d5d08700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d015c1e0 0x7f09d00ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09d5d08700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d015c6a0 con 0x7f09d015be00 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09d5d08700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d01f70c0 con 0x7f09d00af030 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09d5d08700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d006c250 con 0x7f09d015c1e0 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 0x7f09d00c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 0x7f09d00c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58786/0 (socket says 172.21.15.32:58786) 2023-11-24T07:59:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 -- 172.21.15.32:0/2769689808 learned_addr learned my addr 172.21.15.32:0/2769689808 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 -- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d015c1e0 msgr2=0x7f09d00ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d015c1e0 0x7f09d00ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 -- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d00af030 msgr2=0x7f09d015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d00af030 0x7f09d015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.124+0000 7f09ceffd700 1 -- 172.21.15.32:0/2769689808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4009550 con 0x7f09d015be00 2023-11-24T07:59:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.125+0000 7f09ceffd700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 0x7f09d00c8d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009c10 tx=0x7f09c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.125+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40036a0 con 0x7f09d015be00 2023-11-24T07:59:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.125+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c401e070 con 0x7f09d015be00 2023-11-24T07:59:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.125+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d006c530 con 0x7f09d015be00 2023-11-24T07:59:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.125+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4003a40 con 0x7f09d015be00 2023-11-24T07:59:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.125+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d015c8b0 con 0x7f09d015be00 2023-11-24T07:59:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.126+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d015ca40 con 0x7f09d015be00 2023-11-24T07:59:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.126+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09c4003ba0 con 0x7f09d015be00 2023-11-24T07:59:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.127+0000 7f09b7fff700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f09bc06aed0 0x7f09bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.127+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c408a4c0 con 0x7f09d015be00 2023-11-24T07:59:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.127+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f09bc06aed0 0x7f09bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.127+0000 7f09cf7fe700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f09bc06aed0 0x7f09bc06d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f09c0006b20 tx=0x7f09c0006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.131+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09c4055600 con 0x7f09d015be00 2023-11-24T07:59:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.334+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09d0067f60 con 0x7f09d015be00 2023-11-24T07:59:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.335+0000 7f09b7fff700 1 -- 172.21.15.32:0/2769689808 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f09c4058c20 con 0x7f09d015be00 2023-11-24T07:59:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:49.338 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:35.687434+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f09bc06aed0 msgr2=0x7f09bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f09bc06aed0 0x7f09bc06d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f09c0006b20 tx=0x7f09c0006ab0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 msgr2=0x7f09d00c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 0x7f09d00c8d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009c10 tx=0x7f09c4004750 comp rx=0 tx=0).stop 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 shutdown_connections 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f09bc06aed0 0x7f09bc06d390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09d015c1e0 0x7f09d00ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09d00af030 0x7f09d015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 --2- 172.21.15.32:0/2769689808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09d015be00 0x7f09d00c8d50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.337+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 >> 172.21.15.32:0/2769689808 conn(0x7f09d009aa00 msgr2=0x7f09d00b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.338+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 shutdown_connections 2023-11-24T07:59:49.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.338+0000 7f09d5d08700 1 -- 172.21.15.32:0/2769689808 wait complete. 2023-11-24T07:59:49.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:32.221808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:33.067627+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:49.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.478+0000 7fde4c7f7700 1 Processor -- start 2023-11-24T07:59:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.478+0000 7fde4c7f7700 1 -- start start 2023-11-24T07:59:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4c7f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde441587f0 0x7fde4406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4c7f7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4406b530 con 0x7fde440b4370 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4c7f7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde440ae5e0 con 0x7fde4415b370 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4c7f7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4415be30 con 0x7fde441587f0 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde441587f0 0x7fde4406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde441587f0 0x7fde4406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58792/0 (socket says 172.21.15.32:58792) 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4ad94700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fde440b4370 0x7fde440af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60508/0 (socket says 172.21.15.32:60508) 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 -- 172.21.15.32:0/2313134748 learned_addr learned my addr 172.21.15.32:0/2313134748 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 -- 172.21.15.32:0/2313134748 >> v1:172.21.15.123:6789/0 conn(0x7fde4415b370 legacy=0x7fde440b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 -- 172.21.15.32:0/2313134748 >> v1:172.21.15.32:6789/0 conn(0x7fde440b4370 legacy=0x7fde440af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 -- 172.21.15.32:0/2313134748 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4415c110 con 0x7fde441587f0 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde4a593700 1 --2- 172.21.15.32:0/2313134748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde441587f0 0x7fde4406aff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fde400098f0 tx=0x7fde40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e79d507473c243c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde49591700 1 -- 172.21.15.32:0/2313134748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40004230 con 0x7fde441587f0 2023-11-24T07:59:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde49591700 1 -- 172.21.15.32:0/2313134748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40004390 con 0x7fde441587f0 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.479+0000 7fde49591700 1 -- 172.21.15.32:0/2313134748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde400036a0 con 0x7fde441587f0 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2313134748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde441587f0 msgr2=0x7fde4406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2313134748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde441587f0 0x7fde4406aff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fde400098f0 tx=0x7fde40009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2313134748 shutdown_connections 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2313134748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde441587f0 0x7fde4406aff0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2313134748 >> 172.21.15.32:0/2313134748 conn(0x7fde440d0bf0 msgr2=0x7fde440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2313134748 shutdown_connections 2023-11-24T07:59:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.480+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2313134748 wait complete. 2023-11-24T07:59:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.481+0000 7fde4c7f7700 1 Processor -- start 2023-11-24T07:59:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.481+0000 7fde4c7f7700 1 -- start start 2023-11-24T07:59:49.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4c7f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 0x7fde44145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4c7f7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde441587f0 0x7fde440b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4c7f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4415b370 0x7fde440b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4c7f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44146710 con 0x7fde440b4370 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4c7f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415a410 con 0x7fde4415b370 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4c7f7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415a6f0 con 0x7fde441587f0 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 0x7fde44145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 0x7fde44145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58798/0 (socket says 172.21.15.32:58798) 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 -- 172.21.15.32:0/2596280154 learned_addr learned my addr 172.21.15.32:0/2596280154 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4ad94700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4415b370 0x7fde440b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde49d92700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde441587f0 0x7fde440b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 -- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde441587f0 msgr2=0x7fde440b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde441587f0 0x7fde440b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 -- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4415b370 msgr2=0x7fde440b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4415b370 0x7fde440b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4a593700 1 -- 172.21.15.32:0/2596280154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40009550 con 0x7fde440b4370 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.482+0000 7fde4ad94700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4415b370 0x7fde440b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.483+0000 7fde4a593700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 0x7fde44145d70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fde440d1730 tx=0x7fde40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.483+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40003dc0 con 0x7fde440b4370 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.483+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4001f070 con 0x7fde440b4370 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.483+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde441469d0 con 0x7fde440b4370 2023-11-24T07:59:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.483+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000b8a0 con 0x7fde440b4370 2023-11-24T07:59:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.483+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde441462b0 con 0x7fde440b4370 2023-11-24T07:59:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.484+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde44147020 con 0x7fde440b4370 2023-11-24T07:59:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.484+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde40021030 con 0x7fde440b4370 2023-11-24T07:59:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.484+0000 7fde3affd700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fde2806aed0 0x7fde2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.485+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde4008ad90 con 0x7fde440b4370 2023-11-24T07:59:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.485+0000 7fde49d92700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fde2806aed0 0x7fde2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.485+0000 7fde49d92700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fde2806aed0 0x7fde2806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde440741b0 tx=0x7fde34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.489+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde40055e30 con 0x7fde440b4370 2023-11-24T07:59:49.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.687+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde44147020 con 0x7fde440b4370 2023-11-24T07:59:49.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.687+0000 7fde3affd700 1 -- 172.21.15.32:0/2596280154 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde40059450 con 0x7fde440b4370 2023-11-24T07:59:49.689 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fde2806aed0 msgr2=0x7fde2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fde2806aed0 0x7fde2806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde440741b0 tx=0x7fde34008040 comp rx=0 tx=0).stop 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 msgr2=0x7fde44145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 0x7fde44145d70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fde440d1730 tx=0x7fde40004750 comp rx=0 tx=0).stop 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 shutdown_connections 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fde2806aed0 0x7fde2806d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde4415b370 0x7fde440b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde440b4370 0x7fde44145d70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 --2- 172.21.15.32:0/2596280154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fde441587f0 0x7fde440b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 >> 172.21.15.32:0/2596280154 conn(0x7fde440d0bf0 msgr2=0x7fde440b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 shutdown_connections 2023-11-24T07:59:49.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.690+0000 7fde4c7f7700 1 -- 172.21.15.32:0/2596280154 wait complete. 2023-11-24T07:59:49.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.832+0000 7f77e31ec700 1 Processor -- start 2023-11-24T07:59:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.832+0000 7f77e31ec700 1 -- start start 2023-11-24T07:59:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e31ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77dc15e160 0x7f77dc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e31ec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0b3c60 con 0x7f77dc0746a0 2023-11-24T07:59:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e31ec700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0b58c0 con 0x7f77dc0afc90 2023-11-24T07:59:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e31ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc0b4500 con 0x7f77dc15e160 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e0f88700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77dc15e160 0x7f77dc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e0f88700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77dc15e160 0x7f77dc0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58806/0 (socket says 172.21.15.32:58806) 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e0f88700 1 -- 172.21.15.32:0/195226604 learned_addr learned my addr 172.21.15.32:0/195226604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e0f88700 1 -- 172.21.15.32:0/195226604 >> v1:172.21.15.123:6789/0 conn(0x7f77dc0afc90 legacy=0x7f77dc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77e0f88700 1 -- 172.21.15.32:0/195226604 >> v1:172.21.15.32:6789/0 conn(0x7f77dc0746a0 legacy=0x7f77dc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.833+0000 7f77db7fe700 1 -- 172.21.15.32:0/195226604 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027766724 0 0) 0x7f77dc0b3c60 con 0x7f77dc0746a0 2023-11-24T07:59:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e0f88700 1 -- 172.21.15.32:0/195226604 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0b0140 con 0x7f77dc15e160 2023-11-24T07:59:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e0f88700 1 --2- 172.21.15.32:0/195226604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc15e160 0x7f77dc0b5da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f77d00098f0 tx=0x7f77d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48489b130c58dd3c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77db7fe700 1 -- 172.21.15.32:0/195226604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000b8a0 con 0x7f77dc15e160 2023-11-24T07:59:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77db7fe700 1 -- 172.21.15.32:0/195226604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000ba00 con 0x7f77dc15e160 2023-11-24T07:59:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77db7fe700 1 -- 172.21.15.32:0/195226604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000bd90 con 0x7f77dc15e160 2023-11-24T07:59:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 -- 172.21.15.32:0/195226604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc15e160 msgr2=0x7f77dc0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 --2- 172.21.15.32:0/195226604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc15e160 0x7f77dc0b5da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f77d00098f0 tx=0x7f77d0009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 -- 172.21.15.32:0/195226604 shutdown_connections 2023-11-24T07:59:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 --2- 172.21.15.32:0/195226604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc15e160 0x7f77dc0b5da0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 -- 172.21.15.32:0/195226604 >> 172.21.15.32:0/195226604 conn(0x7f77dc0d0bf0 msgr2=0x7f77dc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 -- 172.21.15.32:0/195226604 shutdown_connections 2023-11-24T07:59:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.834+0000 7f77e31ec700 1 -- 172.21.15.32:0/195226604 wait complete. 2023-11-24T07:59:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.836+0000 7f77e31ec700 1 Processor -- start 2023-11-24T07:59:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.836+0000 7f77e31ec700 1 -- start start 2023-11-24T07:59:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.836+0000 7f77e31ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77dc0746a0 0x7f77dc157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.836+0000 7f77e31ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 0x7f77dc155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e31ec700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77dc15e160 0x7f77dc160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e31ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc157100 con 0x7f77dc0afc90 2023-11-24T07:59:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e31ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc155f90 con 0x7f77dc0746a0 2023-11-24T07:59:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e31ec700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc156480 con 0x7f77dc15e160 2023-11-24T07:59:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 0x7f77dc155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 0x7f77dc155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58814/0 (socket says 172.21.15.32:58814) 2023-11-24T07:59:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 -- 172.21.15.32:0/2611370760 learned_addr learned my addr 172.21.15.32:0/2611370760 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e0f88700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77dc0746a0 0x7f77dc157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e1789700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77dc15e160 0x7f77dc160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 -- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77dc15e160 msgr2=0x7f77dc160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77dc15e160 0x7f77dc160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 -- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77dc0746a0 msgr2=0x7f77dc157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77dc0746a0 0x7f77dc157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 -- 172.21.15.32:0/2611370760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d0009550 con 0x7f77dc0afc90 2023-11-24T07:59:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77dbfff700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 0x7f77dc155a50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c510 tx=0x7f77c800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80052d0 con 0x7f77dc0afc90 2023-11-24T07:59:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8005430 con 0x7f77dc0afc90 2023-11-24T07:59:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc156760 con 0x7f77dc0afc90 2023-11-24T07:59:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80038c0 con 0x7f77dc0afc90 2023-11-24T07:59:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.837+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc156d20 con 0x7f77dc0afc90 2023-11-24T07:59:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.838+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc157370 con 0x7f77dc0afc90 2023-11-24T07:59:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.839+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77c8003a20 con 0x7f77dc0afc90 2023-11-24T07:59:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.839+0000 7f77d97fa700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f77c006aed0 0x7f77c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.839+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77c8088970 con 0x7f77dc0afc90 2023-11-24T07:59:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.840+0000 7f77e0f88700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f77c006aed0 0x7f77c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.840+0000 7f77e0f88700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f77c006aed0 0x7f77c006d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77d0014040 tx=0x7f77d001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:49.844+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77c8053a10 con 0x7f77dc0afc90 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.044+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77dc0b51c0 con 0x7f77dc0afc90 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.044+0000 7f77d97fa700 1 -- 172.21.15.32:0/2611370760 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77c8017020 con 0x7f77dc0afc90 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-11-24T07:59:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f77c006aed0 msgr2=0x7f77c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f77c006aed0 0x7f77c006d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77d0014040 tx=0x7f77d001c040 comp rx=0 tx=0).stop 2023-11-24T07:59:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 msgr2=0x7f77dc155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 0x7f77dc155a50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c510 tx=0x7f77c800c820 comp rx=0 tx=0).stop 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 shutdown_connections 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f77c006aed0 0x7f77c006d390 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77dc15e160 0x7f77dc160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77dc0746a0 0x7f77dc157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 --2- 172.21.15.32:0/2611370760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77dc0afc90 0x7f77dc155a50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.046+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 >> 172.21.15.32:0/2611370760 conn(0x7f77dc0d0bf0 msgr2=0x7f77dc15ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.047+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 shutdown_connections 2023-11-24T07:59:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.047+0000 7f77e31ec700 1 -- 172.21.15.32:0/2611370760 wait complete. 2023-11-24T07:59:50.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.186+0000 7f707ca9f700 1 Processor -- start 2023-11-24T07:59:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.186+0000 7f707ca9f700 1 -- start start 2023-11-24T07:59:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f707ca9f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f707815bec0 0x7f707806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f707ca9f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70780d21d0 0x7f70780af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f707ca9f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780cc4a0 con 0x7f70780702e0 2023-11-24T07:59:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f707ca9f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7078070770 con 0x7f70780d21d0 2023-11-24T07:59:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f707ca9f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70780d03e0 con 0x7f707815bec0 2023-11-24T07:59:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70780d21d0 0x7f70780af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70780d21d0 0x7f70780af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58820/0 (socket says 172.21.15.32:58820) 2023-11-24T07:59:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 -- 172.21.15.32:0/4214054749 learned_addr learned my addr 172.21.15.32:0/4214054749 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f707659c700 1 --2- 172.21.15.32:0/4214054749 >> v2:172.21.15.123:3300/0 conn(0x7f707815bec0 0x7f707806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 -- 172.21.15.32:0/4214054749 >> v2:172.21.15.123:3300/0 conn(0x7f707815bec0 msgr2=0x7f707806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 --2- 172.21.15.32:0/4214054749 >> v2:172.21.15.123:3300/0 conn(0x7f707815bec0 0x7f707806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 -- 172.21.15.32:0/4214054749 >> v1:172.21.15.123:6789/0 conn(0x7f70780702e0 legacy=0x7f70780d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.187+0000 7f7076d9d700 1 -- 172.21.15.32:0/4214054749 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707815c2a0 con 0x7f70780d21d0 2023-11-24T07:59:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f7076d9d700 1 --2- 172.21.15.32:0/4214054749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780d21d0 0x7f70780af520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f70780d2d00 tx=0x7f707000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1eb33054925127 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707559a700 1 -- 172.21.15.32:0/4214054749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000e070 con 0x7f70780d21d0 2023-11-24T07:59:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707559a700 1 -- 172.21.15.32:0/4214054749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707000b9c0 con 0x7f70780d21d0 2023-11-24T07:59:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707559a700 1 -- 172.21.15.32:0/4214054749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000bd70 con 0x7f70780d21d0 2023-11-24T07:59:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 -- 172.21.15.32:0/4214054749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780d21d0 msgr2=0x7f70780af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 --2- 172.21.15.32:0/4214054749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780d21d0 0x7f70780af520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f70780d2d00 tx=0x7f707000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 -- 172.21.15.32:0/4214054749 shutdown_connections 2023-11-24T07:59:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 --2- 172.21.15.32:0/4214054749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780d21d0 0x7f70780af520 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 --2- 172.21.15.32:0/4214054749 >> v2:172.21.15.123:3300/0 conn(0x7f707815bec0 0x7f707806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 -- 172.21.15.32:0/4214054749 >> 172.21.15.32:0/4214054749 conn(0x7f70780730a0 msgr2=0x7f707809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 -- 172.21.15.32:0/4214054749 shutdown_connections 2023-11-24T07:59:50.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.188+0000 7f707ca9f700 1 -- 172.21.15.32:0/4214054749 wait complete. 2023-11-24T07:59:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.190+0000 7f707ca9f700 1 Processor -- start 2023-11-24T07:59:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.190+0000 7f707ca9f700 1 -- start start 2023-11-24T07:59:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707ca9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 0x7f70780b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707ca9f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70780d21d0 0x7f70780c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707ca9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f707815bec0 0x7f70780ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707ca9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c85a0 con 0x7f70780702e0 2023-11-24T07:59:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707ca9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c82c0 con 0x7f707815bec0 2023-11-24T07:59:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707ca9f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780b1840 con 0x7f70780d21d0 2023-11-24T07:59:50.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 0x7f70780b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 0x7f70780b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58824/0 (socket says 172.21.15.32:58824) 2023-11-24T07:59:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 -- 172.21.15.32:0/2884572171 learned_addr learned my addr 172.21.15.32:0/2884572171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f7075d9b700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70780d21d0 0x7f70780c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f7076d9d700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f707815bec0 0x7f70780ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 -- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70780d21d0 msgr2=0x7f70780c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70780d21d0 0x7f70780c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 -- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f707815bec0 msgr2=0x7f70780ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f707815bec0 0x7f70780ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.191+0000 7f707659c700 1 -- 172.21.15.32:0/2884572171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070009b30 con 0x7f70780702e0 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.192+0000 7f707659c700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 0x7f70780b2320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f706c00d800 tx=0x7f706c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.192+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0125f0 con 0x7f70780702e0 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.192+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c012750 con 0x7f70780702e0 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.192+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780b1b20 con 0x7f70780702e0 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.192+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c012b00 con 0x7f70780702e0 2023-11-24T07:59:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.192+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780b4400 con 0x7f70780702e0 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.193+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70780b16a0 con 0x7f70780702e0 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.193+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f706c012c60 con 0x7f70780702e0 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.194+0000 7f7066ffd700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f705806aed0 0x7f705806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.194+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f706c0899d0 con 0x7f70780702e0 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.194+0000 7f7075d9b700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f705806aed0 0x7f705806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.194+0000 7f7075d9b700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f705806aed0 0x7f705806d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f707815ec00 tx=0x7f7068009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.198+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70780b16a0 con 0x7f70780702e0 2023-11-24T07:59:50.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.397+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70780b16a0 con 0x7f70780702e0 2023-11-24T07:59:50.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.397+0000 7f7066ffd700 1 -- 172.21.15.32:0/2884572171 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70780b16a0 con 0x7f70780702e0 2023-11-24T07:59:50.399 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-11-24T07:59:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f705806aed0 msgr2=0x7f705806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f705806aed0 0x7f705806d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f707815ec00 tx=0x7f7068009040 comp rx=0 tx=0).stop 2023-11-24T07:59:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 msgr2=0x7f70780b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 0x7f70780b2320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f706c00d800 tx=0x7f706c00dbc0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 shutdown_connections 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f705806aed0 0x7f705806d390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f707815bec0 0x7f70780ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70780702e0 0x7f70780b2320 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 --2- 172.21.15.32:0/2884572171 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70780d21d0 0x7f70780c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 >> 172.21.15.32:0/2884572171 conn(0x7f70780730a0 msgr2=0x7f707809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 shutdown_connections 2023-11-24T07:59:50.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.399+0000 7f707ca9f700 1 -- 172.21.15.32:0/2884572171 wait complete. 2023-11-24T07:59:50.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:50.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.537+0000 7f77aa582700 1 Processor -- start 2023-11-24T07:59:50.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.537+0000 7f77aa582700 1 -- start start 2023-11-24T07:59:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.538+0000 7f77aa582700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.538+0000 7f77aa582700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77a40b1940 0x7f77a40b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.538+0000 7f77aa582700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40af8c0 con 0x7f77a415afc0 2023-11-24T07:59:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.538+0000 7f77aa582700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40b0790 con 0x7f77a40b1940 2023-11-24T07:59:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.538+0000 7f77aa582700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40b1dd0 con 0x7f77a40b0bb0 2023-11-24T07:59:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a8b1f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77a40b1940 0x7f77a40b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:54086/0 (socket says 172.21.15.32:54086) 2023-11-24T07:59:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 -- 172.21.15.32:0/1818506785 learned_addr learned my addr 172.21.15.32:0/1818506785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a2ffd700 1 -- 172.21.15.32:0/1818506785 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295506570 0 0) 0x7f77a40af8c0 con 0x7f77a415afc0 2023-11-24T07:59:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 -- 172.21.15.32:0/1818506785 >> v2:172.21.15.123:3300/0 conn(0x7f77a40b1940 msgr2=0x7f77a40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 --2- 172.21.15.32:0/1818506785 >> v2:172.21.15.123:3300/0 conn(0x7f77a40b1940 0x7f77a40b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 -- 172.21.15.32:0/1818506785 >> v1:172.21.15.123:6789/0 conn(0x7f77a415afc0 legacy=0x7f77a40cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 -- 172.21.15.32:0/1818506785 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40b1030 con 0x7f77a40b0bb0 2023-11-24T07:59:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a37fe700 1 --2- 172.21.15.32:0/1818506785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40b1400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77980098f0 tx=0x7f7798009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1937c676d4e01d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.539+0000 7f77a2ffd700 1 -- 172.21.15.32:0/1818506785 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800f8a0 con 0x7f77a40b0bb0 2023-11-24T07:59:50.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77a2ffd700 1 -- 172.21.15.32:0/1818506785 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779800fa00 con 0x7f77a40b0bb0 2023-11-24T07:59:50.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77a2ffd700 1 -- 172.21.15.32:0/1818506785 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800fd90 con 0x7f77a40b0bb0 2023-11-24T07:59:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 -- 172.21.15.32:0/1818506785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 msgr2=0x7f77a40b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 --2- 172.21.15.32:0/1818506785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40b1400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77980098f0 tx=0x7f7798009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 -- 172.21.15.32:0/1818506785 shutdown_connections 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 --2- 172.21.15.32:0/1818506785 >> v2:172.21.15.123:3300/0 conn(0x7f77a40b1940 0x7f77a40b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 --2- 172.21.15.32:0/1818506785 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40b1400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 -- 172.21.15.32:0/1818506785 >> 172.21.15.32:0/1818506785 conn(0x7f77a40b57b0 msgr2=0x7f77a40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 -- 172.21.15.32:0/1818506785 shutdown_connections 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.540+0000 7f77aa582700 1 -- 172.21.15.32:0/1818506785 wait complete. 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.541+0000 7f77aa582700 1 Processor -- start 2023-11-24T07:59:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.541+0000 7f77aa582700 1 -- start start 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77aa582700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77aa582700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40b1940 0x7f77a40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77aa582700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 0x7f77a414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77aa582700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40af070 con 0x7f77a415afc0 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77aa582700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41574d0 con 0x7f77a40b1940 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77aa582700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4155460 con 0x7f77a40b0bb0 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77a8b1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 0x7f77a414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77a8b1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 0x7f77a414d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58842/0 (socket says 172.21.15.32:58842) 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77a8b1f700 1 -- 172.21.15.32:0/1321356375 learned_addr learned my addr 172.21.15.32:0/1321356375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.542+0000 7f77a3fff700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a37fe700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40b1940 0x7f77a40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a8b1f700 1 -- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 msgr2=0x7f77a40adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a8b1f700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a8b1f700 1 -- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40b1940 msgr2=0x7f77a40c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a8b1f700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40b1940 0x7f77a40c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a8b1f700 1 -- 172.21.15.32:0/1321356375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798009550 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a8b1f700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 0x7f77a414d6c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f779c00efc0 tx=0x7f779c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00cd40 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00f040 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a415b760 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c0146c0 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.543+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a40aec90 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.544+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4067f60 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.544+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f779c014820 con 0x7f77a415afc0 2023-11-24T07:59:50.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.545+0000 7f77a0ff9700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f779006aed0 0x7f779006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.545+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f779c089f40 con 0x7f77a415afc0 2023-11-24T07:59:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.545+0000 7f77a3fff700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f779006aed0 0x7f779006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.546+0000 7f77a3fff700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f779006aed0 0x7f779006d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7794000f60 tx=0x7f7794009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.550+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f779c055000 con 0x7f77a415afc0 2023-11-24T07:59:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.798+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77a40715d0 con 0x7f77a415afc0 2023-11-24T07:59:50.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.798+0000 7f77a0ff9700 1 -- 172.21.15.32:0/1321356375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f779c00cea0 con 0x7f77a415afc0 2023-11-24T07:59:50.800 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:50.800 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:34.106005+0000","modified":"2023-11-24T07:59:37.114690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/598023464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":598023464},{"type":"v1","addr":"172.21.15.32:6835","nonce":598023464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f779006aed0 msgr2=0x7f779006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f779006aed0 0x7f779006d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7794000f60 tx=0x7f7794009040 comp rx=0 tx=0).stop 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 msgr2=0x7f77a414d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 0x7f77a414d6c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f779c00efc0 tx=0x7f779c00c3a0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 shutdown_connections 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f779006aed0 0x7f779006d390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77a415afc0 0x7f77a414d6c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a40b0bb0 0x7f77a40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 --2- 172.21.15.32:0/1321356375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40b1940 0x7f77a40c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 >> 172.21.15.32:0/1321356375 conn(0x7f77a40b57b0 msgr2=0x7f77a4061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 shutdown_connections 2023-11-24T07:59:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.801+0000 7f77aa582700 1 -- 172.21.15.32:0/1321356375 wait complete. 2023-11-24T07:59:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2023-11-24T07:59:50.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.938+0000 7f50569a8700 1 Processor -- start 2023-11-24T07:59:50.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.938+0000 7f50569a8700 1 -- start start 2023-11-24T07:59:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f50569a8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f50501446a0 0x7f50500b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f50569a8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 0x7f505006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f50569a8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500af950 con 0x7f505006c540 2023-11-24T07:59:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f50569a8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5050144a80 con 0x7f50501446a0 2023-11-24T07:59:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f50569a8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505006c9b0 con 0x7f50500adf00 2023-11-24T07:59:50.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f5054f45700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f505006c540 0x7f505006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60562/0 (socket says 172.21.15.32:60562) 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f5054f45700 1 -- 172.21.15.32:0/2620831552 learned_addr learned my addr 172.21.15.32:0/2620831552 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f504ffff700 1 --2- 172.21.15.32:0/2620831552 >> v2:172.21.15.123:3300/0 conn(0x7f50501446a0 0x7f50500b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f504f7fe700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 0x7f505006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241864592 0 0) 0x7f50500af950 con 0x7f505006c540 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c0035b0 con 0x7f505006c540 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807354653 0 0) 0x7f503c0035b0 con 0x7f505006c540 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.939+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 msgr2=0x7f505006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 0x7f505006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 >> v2:172.21.15.123:3300/0 conn(0x7f50501446a0 msgr2=0x7f50500b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 --2- 172.21.15.32:0/2620831552 >> v2:172.21.15.123:3300/0 conn(0x7f50501446a0 0x7f50500b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500afc30 con 0x7f505006c540 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f50480019f0 con 0x7f505006c540 2023-11-24T07:59:50.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 >> v1:172.21.15.32:6789/0 conn(0x7f505006c540 legacy=0x7f505006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503c003a30 0x7f503c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 0x7f503c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503c004d90 0x7f503c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5050144a80 con 0x7f503c004d90 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505006c9b0 con 0x7f503c003a30 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c00ba10 con 0x7f50500adf00 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503c004d90 0x7f503c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504ffff700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503c003a30 0x7f503c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504f7fe700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 0x7f503c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 -- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 msgr2=0x7f503c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500adf00 0x7f503c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 -- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503c003a30 msgr2=0x7f503c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f503c003a30 0x7f503c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 -- 172.21.15.32:0/2620831552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0035b0 con 0x7f503c004d90 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f5054f45700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503c004d90 0x7f503c005210 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f504800a860 tx=0x7f503c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc063cbb8ae55559 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504800a2f0 con 0x7f503c004d90 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5048012620 con 0x7f503c004d90 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.940+0000 7f504effd700 1 -- 172.21.15.32:0/2620831552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480129d0 con 0x7f503c004d90 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f50569a8700 1 -- 172.21.15.32:0/2620831552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503c004d90 msgr2=0x7f503c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f50569a8700 1 --2- 172.21.15.32:0/2620831552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f503c004d90 0x7f503c005210 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f504800a860 tx=0x7f503c005880 comp rx=0 tx=0).stop 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f5054f45700 1 -- 172.21.15.32:0/2620831552 reap_dead start 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f50569a8700 1 -- 172.21.15.32:0/2620831552 shutdown_connections 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f50569a8700 1 -- 172.21.15.32:0/2620831552 >> 172.21.15.32:0/2620831552 conn(0x7f5050071b90 msgr2=0x7f5050071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f50569a8700 1 -- 172.21.15.32:0/2620831552 shutdown_connections 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.941+0000 7f50569a8700 1 -- 172.21.15.32:0/2620831552 wait complete. 2023-11-24T07:59:50.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.942+0000 7f50569a8700 1 Processor -- start 2023-11-24T07:59:50.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.942+0000 7f50569a8700 1 -- start start 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f50569a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 0x7f5050159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f50569a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5050159da0 0x7f50500cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f50569a8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500cc3e0 0x7f50500ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f50569a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5050159150 con 0x7f505015dd90 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f50569a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505015b310 con 0x7f5050159da0 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f50569a8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505015a220 con 0x7f50500cc3e0 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f504ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 0x7f5050159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f504ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 0x7f5050159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58860/0 (socket says 172.21.15.32:58860) 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f504ffff700 1 -- 172.21.15.32:0/4249079286 learned_addr learned my addr 172.21.15.32:0/4249079286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f5054f45700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500cc3e0 0x7f50500ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.943+0000 7f504f7fe700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5050159da0 0x7f50500cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504ffff700 1 -- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500cc3e0 msgr2=0x7f50500ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504ffff700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500cc3e0 0x7f50500ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504ffff700 1 -- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5050159da0 msgr2=0x7f50500cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504ffff700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5050159da0 0x7f50500cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504ffff700 1 -- 172.21.15.32:0/4249079286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500cc870 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504ffff700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 0x7f5050159860 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5038011aa0 tx=0x7f5038011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038011ed0 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503800f940 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505013d840 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503800fd10 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.944+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505015b520 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.945+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505015be50 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.946+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50380263f0 con 0x7f505015dd90 2023-11-24T07:59:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.946+0000 7f504cff9700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f503006ae60 0x7f503006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.946+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5038089d20 con 0x7f505015dd90 2023-11-24T07:59:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.946+0000 7f504f7fe700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f503006ae60 0x7f503006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.947+0000 7f504f7fe700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f503006ae60 0x7f503006d320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f505006eef0 tx=0x7f5040009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:50.950+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5038055000 con 0x7f505015dd90 2023-11-24T07:59:51.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.152+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50500753f0 con 0x7f505015dd90 2023-11-24T07:59:51.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.153+0000 7f504cff9700 1 -- 172.21.15.32:0/4249079286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f5038058620 con 0x7f505015dd90 2023-11-24T07:59:51.155 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:51.155 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:35.687434+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f503006ae60 msgr2=0x7f503006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f503006ae60 0x7f503006d320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f505006eef0 tx=0x7f5040009210 comp rx=0 tx=0).stop 2023-11-24T07:59:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 msgr2=0x7f5050159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 0x7f5050159860 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5038011aa0 tx=0x7f5038011db0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 shutdown_connections 2023-11-24T07:59:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f503006ae60 0x7f503006d320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50500cc3e0 0x7f50500ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505015dd90 0x7f5050159860 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 --2- 172.21.15.32:0/4249079286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5050159da0 0x7f50500cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 >> 172.21.15.32:0/4249079286 conn(0x7f5050071b90 msgr2=0x7f50500b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 shutdown_connections 2023-11-24T07:59:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.155+0000 7f50569a8700 1 -- 172.21.15.32:0/4249079286 wait complete. 2023-11-24T07:59:51.169 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:59:51.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.293+0000 7fa0f8fef700 1 Processor -- start 2023-11-24T07:59:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.293+0000 7fa0f8fef700 1 -- start start 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f8fef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f415bef0 0x7fa0f415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f8fef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0f41624d0 0x7fa0f4162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f8fef700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f406dd80 con 0x7fa0f40afe70 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f8fef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f415f880 con 0x7fa0f41624d0 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f8fef700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f415ed90 con 0x7fa0f415bef0 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0f41624d0 0x7fa0f4162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0f41624d0 0x7fa0f4162960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58870/0 (socket says 172.21.15.32:58870) 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 -- 172.21.15.32:0/3569820568 learned_addr learned my addr 172.21.15.32:0/3569820568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f259c700 1 --2- 172.21.15.32:0/3569820568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f415bef0 0x7fa0f415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 -- 172.21.15.32:0/3569820568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f415bef0 msgr2=0x7fa0f415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 --2- 172.21.15.32:0/3569820568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f415bef0 0x7fa0f415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 -- 172.21.15.32:0/3569820568 >> v1:172.21.15.123:6789/0 conn(0x7fa0f40afe70 legacy=0x7fa0f40b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 -- 172.21.15.32:0/3569820568 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f415c2d0 con 0x7fa0f41624d0 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.294+0000 7fa0f2d9d700 1 --2- 172.21.15.32:0/3569820568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f41624d0 0x7fa0f4162960 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009e50 tx=0x7fa0ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb989791cfae53d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f159a700 1 -- 172.21.15.32:0/3569820568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00e070 con 0x7fa0f41624d0 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f159a700 1 -- 172.21.15.32:0/3569820568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00b910 con 0x7fa0f41624d0 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f159a700 1 -- 172.21.15.32:0/3569820568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00bcc0 con 0x7fa0f41624d0 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 -- 172.21.15.32:0/3569820568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f41624d0 msgr2=0x7fa0f4162960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/3569820568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f41624d0 0x7fa0f4162960 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009e50 tx=0x7fa0ec00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 -- 172.21.15.32:0/3569820568 shutdown_connections 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/3569820568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f41624d0 0x7fa0f4162960 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/3569820568 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f415bef0 0x7fa0f415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 -- 172.21.15.32:0/3569820568 >> 172.21.15.32:0/3569820568 conn(0x7fa0f4071ac0 msgr2=0x7fa0f4071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 -- 172.21.15.32:0/3569820568 shutdown_connections 2023-11-24T07:59:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.295+0000 7fa0f8fef700 1 -- 172.21.15.32:0/3569820568 wait complete. 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 Processor -- start 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 -- start start 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f40afe70 0x7fa0f415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 0x7fa0f4156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0f41624d0 0x7fa0f4148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41e8b40 con 0x7fa0f415bef0 2023-11-24T07:59:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41e8e20 con 0x7fa0f41624d0 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.297+0000 7fa0f8fef700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41e9100 con 0x7fa0f40afe70 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 0x7fa0f4156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 0x7fa0f4156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58872/0 (socket says 172.21.15.32:58872) 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 -- 172.21.15.32:0/2566845461 learned_addr learned my addr 172.21.15.32:0/2566845461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f2d9d700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0f41624d0 0x7fa0f4148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f259c700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f40afe70 0x7fa0f415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 -- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f40afe70 msgr2=0x7fa0f415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f40afe70 0x7fa0f415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 -- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0f41624d0 msgr2=0x7fa0f4148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0f41624d0 0x7fa0f4148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 -- 172.21.15.32:0/2566845461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec009b30 con 0x7fa0f415bef0 2023-11-24T07:59:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f1d9b700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 0x7fa0f4156d50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800c450 tx=0x7fa0e800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80052d0 con 0x7fa0f415bef0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8005430 con 0x7fa0f415bef0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f41e93e0 con 0x7fa0f415bef0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80038c0 con 0x7fa0f415bef0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.298+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f41e8720 con 0x7fa0f415bef0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.299+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f41e8910 con 0x7fa0f415bef0 2023-11-24T07:59:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.300+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0e8003a20 con 0x7fa0f415bef0 2023-11-24T07:59:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.300+0000 7fa0e2ffd700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0d406aed0 0x7fa0d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.300+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0e80888f0 con 0x7fa0f415bef0 2023-11-24T07:59:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.300+0000 7fa0f259c700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0d406aed0 0x7fa0d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.301+0000 7fa0f259c700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0d406aed0 0x7fa0d406d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009be0 tx=0x7fa0e4009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.304+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0e80539a0 con 0x7fa0f415bef0 2023-11-24T07:59:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.553+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0f406db50 con 0x7fa0f415bef0 2023-11-24T07:59:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.553+0000 7fa0e2ffd700 1 -- 172.21.15.32:0/2566845461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fa0e8017090 con 0x7fa0f415bef0 2023-11-24T07:59:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:51.555 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:34.106005+0000","modified":"2023-11-24T07:59:37.114690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/598023464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":598023464},{"type":"v1","addr":"172.21.15.32:6835","nonce":598023464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0d406aed0 msgr2=0x7fa0d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0d406aed0 0x7fa0d406d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009be0 tx=0x7fa0e4009210 comp rx=0 tx=0).stop 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 msgr2=0x7fa0f4156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 0x7fa0f4156d50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800c450 tx=0x7fa0e800c760 comp rx=0 tx=0).stop 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 shutdown_connections 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa0d406aed0 0x7fa0d406d390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0f41624d0 0x7fa0f4148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0f40afe70 0x7fa0f415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 --2- 172.21.15.32:0/2566845461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f415bef0 0x7fa0f4156d50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 >> 172.21.15.32:0/2566845461 conn(0x7fa0f4071ac0 msgr2=0x7fa0f40b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 shutdown_connections 2023-11-24T07:59:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.555+0000 7fa0f8fef700 1 -- 172.21.15.32:0/2566845461 wait complete. 2023-11-24T07:59:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2023-11-24T07:59:51.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:51.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.692+0000 7fd597891700 1 Processor -- start 2023-11-24T07:59:51.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.692+0000 7fd597891700 1 -- start start 2023-11-24T07:59:51.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd597891700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd59015b8d0 0x7fd59015e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd597891700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5900b3520 con 0x7fd59015bcb0 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd597891700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5900ae230 con 0x7fd590071a50 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd597891700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5900ae490 con 0x7fd59015b8d0 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd59015b8d0 0x7fd59015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd59015b8d0 0x7fd59015e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58880/0 (socket says 172.21.15.32:58880) 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd594e2c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd59015bcb0 0x7fd590071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60592/0 (socket says 172.21.15.32:60592) 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 -- 172.21.15.32:0/3991993842 learned_addr learned my addr 172.21.15.32:0/3991993842 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:51.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 -- 172.21.15.32:0/3991993842 >> v1:172.21.15.123:6789/0 conn(0x7fd590071a50 legacy=0x7fd5900ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 -- 172.21.15.32:0/3991993842 >> v1:172.21.15.32:6789/0 conn(0x7fd59015bcb0 legacy=0x7fd590071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd587fff700 1 -- 172.21.15.32:0/3991993842 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277182591 0 0) 0x7fd5900b3520 con 0x7fd59015bcb0 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 -- 172.21.15.32:0/3991993842 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd590162210 con 0x7fd59015b8d0 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.693+0000 7fd59562d700 1 --2- 172.21.15.32:0/3991993842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59015b8d0 0x7fd59015e520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0098f0 tx=0x7fd58c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebd54d0e5253e70 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd587fff700 1 -- 172.21.15.32:0/3991993842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00b8a0 con 0x7fd59015b8d0 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd587fff700 1 -- 172.21.15.32:0/3991993842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c00ba00 con 0x7fd59015b8d0 2023-11-24T07:59:51.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd587fff700 1 -- 172.21.15.32:0/3991993842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00bd90 con 0x7fd59015b8d0 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 -- 172.21.15.32:0/3991993842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59015b8d0 msgr2=0x7fd59015e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 --2- 172.21.15.32:0/3991993842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59015b8d0 0x7fd59015e520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0098f0 tx=0x7fd58c009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 -- 172.21.15.32:0/3991993842 shutdown_connections 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 --2- 172.21.15.32:0/3991993842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd59015b8d0 0x7fd59015e520 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 -- 172.21.15.32:0/3991993842 >> 172.21.15.32:0/3991993842 conn(0x7fd5900743b0 msgr2=0x7fd59006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 -- 172.21.15.32:0/3991993842 shutdown_connections 2023-11-24T07:59:51.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.694+0000 7fd597891700 1 -- 172.21.15.32:0/3991993842 wait complete. 2023-11-24T07:59:51.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.695+0000 7fd597891700 1 Processor -- start 2023-11-24T07:59:51.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 -- start start 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 0x7fd590159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd59015b8d0 0x7fd5901499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59015bcb0 0x7fd590157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590153f60 con 0x7fd590071a50 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590157ff0 con 0x7fd59015b8d0 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd597891700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901582d0 con 0x7fd59015bcb0 2023-11-24T07:59:51.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd59562d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 0x7fd590159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd59562d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 0x7fd590159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58886/0 (socket says 172.21.15.32:58886) 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd59562d700 1 -- 172.21.15.32:0/3946350375 learned_addr learned my addr 172.21.15.32:0/3946350375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.696+0000 7fd595e2e700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59015bcb0 0x7fd590157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd594e2c700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd59015b8d0 0x7fd5901499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd59562d700 1 -- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59015bcb0 msgr2=0x7fd590157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd59562d700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59015bcb0 0x7fd590157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd59562d700 1 -- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd59015b8d0 msgr2=0x7fd5901499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd59562d700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd59015b8d0 0x7fd5901499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd59562d700 1 -- 172.21.15.32:0/3946350375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c009550 con 0x7fd590071a50 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd595e2e700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59015bcb0 0x7fd590157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd594e2c700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd59015b8d0 0x7fd5901499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd59562d700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 0x7fd590159090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd58c000c00 tx=0x7fd58c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011a70 con 0x7fd590071a50 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c01f070 con 0x7fd590071a50 2023-11-24T07:59:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011dd0 con 0x7fd590071a50 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901585b0 con 0x7fd590071a50 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.697+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd590158b70 con 0x7fd590071a50 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.698+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5900753f0 con 0x7fd590071a50 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.699+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd58c026480 con 0x7fd590071a50 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.699+0000 7fd585ffb700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd57406aed0 0x7fd57406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.699+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd58c021070 con 0x7fd590071a50 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.699+0000 7fd594e2c700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd57406aed0 0x7fd57406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.700+0000 7fd594e2c700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd57406aed0 0x7fd57406d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd5901553b0 tx=0x7fd580007e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.703+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd58c0561d0 con 0x7fd590071a50 2023-11-24T07:59:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.953+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd590158ee0 con 0x7fd590071a50 2023-11-24T07:59:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.954+0000 7fd585ffb700 1 -- 172.21.15.32:0/3946350375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd58c0597f0 con 0x7fd590071a50 2023-11-24T07:59:51.956 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:51.956 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:34.106005+0000","modified":"2023-11-24T07:59:37.114690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6835/598023464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":598023464},{"type":"v1","addr":"172.21.15.32:6835","nonce":598023464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:59:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd57406aed0 msgr2=0x7fd57406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd57406aed0 0x7fd57406d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd5901553b0 tx=0x7fd580007e50 comp rx=0 tx=0).stop 2023-11-24T07:59:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 msgr2=0x7fd590159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 0x7fd590159090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd58c000c00 tx=0x7fd58c004750 comp rx=0 tx=0).stop 2023-11-24T07:59:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 shutdown_connections 2023-11-24T07:59:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fd57406aed0 0x7fd57406d390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd59015bcb0 0x7fd590157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd590071a50 0x7fd590159090 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 --2- 172.21.15.32:0/3946350375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd59015b8d0 0x7fd5901499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 >> 172.21.15.32:0/3946350375 conn(0x7fd5900743b0 msgr2=0x7fd5900cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 shutdown_connections 2023-11-24T07:59:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:51.956+0000 7fd597891700 1 -- 172.21.15.32:0/3946350375 wait complete. 2023-11-24T07:59:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2023-11-24T07:59:51.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:52.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.094+0000 7fcab9e83700 1 Processor -- start 2023-11-24T07:59:52.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.094+0000 7fcab9e83700 1 -- start start 2023-11-24T07:59:52.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab9e83700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcab415c030 0x7fcab40d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab9e83700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcab40ae910 0x7fcab40d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab9e83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cc8c0 0x7fcab40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab9e83700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40cd5d0 con 0x7fcab40ae910 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab9e83700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40cd870 con 0x7fcab415c030 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab9e83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40cdb50 con 0x7fcab40cc8c0 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcab40ae910 0x7fcab40d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcab40ae910 0x7fcab40d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58896/0 (socket says 172.21.15.32:58896) 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab2ffd700 1 -- 172.21.15.32:0/4246076772 learned_addr learned my addr 172.21.15.32:0/4246076772 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab37fe700 1 --2- 172.21.15.32:0/4246076772 >> v2:172.21.15.123:3300/0 conn(0x7fcab415c030 0x7fcab40d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.095+0000 7fcab3fff700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cc8c0 0x7fcab40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab2ffd700 1 -- 172.21.15.32:0/4246076772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cc8c0 msgr2=0x7fcab40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab2ffd700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cc8c0 0x7fcab40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab2ffd700 1 -- 172.21.15.32:0/4246076772 >> v2:172.21.15.123:3300/0 conn(0x7fcab415c030 msgr2=0x7fcab40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab2ffd700 1 --2- 172.21.15.32:0/4246076772 >> v2:172.21.15.123:3300/0 conn(0x7fcab415c030 0x7fcab40d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab2ffd700 1 -- 172.21.15.32:0/4246076772 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40cde30 con 0x7fcab40ae910 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab3fff700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cc8c0 0x7fcab40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab2ffd700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab40d05b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcab4073610 tx=0x7fca9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d5f2f76b2f9957 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab27fc700 1 -- 172.21.15.32:0/4246076772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00b870 con 0x7fcab40ae910 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab27fc700 1 -- 172.21.15.32:0/4246076772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c00b9d0 con 0x7fcab40ae910 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab27fc700 1 -- 172.21.15.32:0/4246076772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00bd60 con 0x7fcab40ae910 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 -- 172.21.15.32:0/4246076772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 msgr2=0x7fcab40d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab40d05b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcab4073610 tx=0x7fca9c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 -- 172.21.15.32:0/4246076772 shutdown_connections 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cc8c0 0x7fcab40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 --2- 172.21.15.32:0/4246076772 >> v2:172.21.15.123:3300/0 conn(0x7fcab415c030 0x7fcab40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 --2- 172.21.15.32:0/4246076772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab40d05b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.096+0000 7fcab9e83700 1 -- 172.21.15.32:0/4246076772 >> 172.21.15.32:0/4246076772 conn(0x7fcab40730a0 msgr2=0x7fcab409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.097+0000 7fcab9e83700 1 -- 172.21.15.32:0/4246076772 shutdown_connections 2023-11-24T07:59:52.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.097+0000 7fcab9e83700 1 -- 172.21.15.32:0/4246076772 wait complete. 2023-11-24T07:59:52.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.098+0000 7fcab9e83700 1 Processor -- start 2023-11-24T07:59:52.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.098+0000 7fcab9e83700 1 -- start start 2023-11-24T07:59:52.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab9e83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab41f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab9e83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab40cc8c0 0x7fcab41f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab9e83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cb000 0x7fcab40cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab9e83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40cb9d0 con 0x7fcab40ae910 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab9e83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41f5a50 con 0x7fcab40cc8c0 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab9e83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab415dee0 con 0x7fcab40cb000 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab41f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab41f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58900/0 (socket says 172.21.15.32:58900) 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.099+0000 7fcab37fe700 1 -- 172.21.15.32:0/702364735 learned_addr learned my addr 172.21.15.32:0/702364735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab3fff700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cb000 0x7fcab40cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab2ffd700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab40cc8c0 0x7fcab41f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab37fe700 1 -- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cb000 msgr2=0x7fcab40cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab37fe700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cb000 0x7fcab40cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab37fe700 1 -- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab40cc8c0 msgr2=0x7fcab41f0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab37fe700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab40cc8c0 0x7fcab41f0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab37fe700 1 -- 172.21.15.32:0/702364735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c009550 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab3fff700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cb000 0x7fcab40cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab37fe700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab41f5510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400d800 tx=0x7fcaa400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40125f0 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4012750 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab415e1c0 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4012b00 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.100+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab415e780 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.101+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab40751e0 con 0x7fcab40ae910 2023-11-24T07:59:52.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.102+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaa4012c60 con 0x7fcab40ae910 2023-11-24T07:59:52.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.102+0000 7fcaabfff700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fca9406aed0 0x7fca9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.102+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaa4089a60 con 0x7fcab40ae910 2023-11-24T07:59:52.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.102+0000 7fcab2ffd700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fca9406aed0 0x7fca9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.103+0000 7fcab2ffd700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fca9406aed0 0x7fca9406d390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcab40aed30 tx=0x7fca9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.106+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaa4054a90 con 0x7fcab40ae910 2023-11-24T07:59:52.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.307+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcab40b3ca0 con 0x7fcab40ae910 2023-11-24T07:59:52.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.308+0000 7fcaabfff700 1 -- 172.21.15.32:0/702364735 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fcaa40580b0 con 0x7fcab40ae910 2023-11-24T07:59:52.310 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T07:59:52.310 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:35.687434+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:32.221808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:33.067627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:52.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fca9406aed0 msgr2=0x7fca9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fca9406aed0 0x7fca9406d390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcab40aed30 tx=0x7fca9c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 msgr2=0x7fcab41f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab41f5510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400d800 tx=0x7fcaa400dbc0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 shutdown_connections 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fca9406aed0 0x7fca9406d390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcab40cb000 0x7fcab40cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab40ae910 0x7fcab41f5510 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 --2- 172.21.15.32:0/702364735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab40cc8c0 0x7fcab41f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 >> 172.21.15.32:0/702364735 conn(0x7fcab40730a0 msgr2=0x7fcab40aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 shutdown_connections 2023-11-24T07:59:52.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.311+0000 7fcab9e83700 1 -- 172.21.15.32:0/702364735 wait complete. 2023-11-24T07:59:52.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:59:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.449+0000 7f9e40a01700 1 Processor -- start 2023-11-24T07:59:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.449+0000 7f9e40a01700 1 -- start start 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e40a01700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9e3c1446a0 0x7f9e3c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e40a01700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e3c0adf00 0x7f9e3c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e40a01700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0af950 con 0x7f9e3c06c540 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e40a01700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c06c9f0 con 0x7f9e3c0adf00 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e40a01700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c144a80 con 0x7f9e3c1446a0 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e3c0adf00 0x7f9e3c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e3c0adf00 0x7f9e3c06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58908/0 (socket says 172.21.15.32:58908) 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 -- 172.21.15.32:0/3835163993 learned_addr learned my addr 172.21.15.32:0/3835163993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e3a59c700 1 --2- 172.21.15.32:0/3835163993 >> v2:172.21.15.123:3300/0 conn(0x7f9e3c1446a0 0x7f9e3c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 -- 172.21.15.32:0/3835163993 >> v2:172.21.15.123:3300/0 conn(0x7f9e3c1446a0 msgr2=0x7f9e3c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 --2- 172.21.15.32:0/3835163993 >> v2:172.21.15.123:3300/0 conn(0x7f9e3c1446a0 0x7f9e3c0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 -- 172.21.15.32:0/3835163993 >> v1:172.21.15.32:6789/0 conn(0x7f9e3c06c540 legacy=0x7f9e3c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 -- 172.21.15.32:0/3835163993 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0afc30 con 0x7f9e3c0adf00 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e3a59c700 1 --2- 172.21.15.32:0/3835163993 >> v2:172.21.15.123:3300/0 conn(0x7f9e3c1446a0 0x7f9e3c0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.450+0000 7f9e39d9b700 1 --2- 172.21.15.32:0/3835163993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c06c000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0729a0 tx=0x7f9e2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c3c71f6d800b01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e3959a700 1 -- 172.21.15.32:0/3835163993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00b860 con 0x7f9e3c0adf00 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e3959a700 1 -- 172.21.15.32:0/3835163993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c00b9c0 con 0x7f9e3c0adf00 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e3959a700 1 -- 172.21.15.32:0/3835163993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00bd50 con 0x7f9e3c0adf00 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 -- 172.21.15.32:0/3835163993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0adf00 msgr2=0x7f9e3c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 --2- 172.21.15.32:0/3835163993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c06c000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0729a0 tx=0x7f9e2c009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 -- 172.21.15.32:0/3835163993 shutdown_connections 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 --2- 172.21.15.32:0/3835163993 >> v2:172.21.15.123:3300/0 conn(0x7f9e3c1446a0 0x7f9e3c0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 --2- 172.21.15.32:0/3835163993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c06c000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 -- 172.21.15.32:0/3835163993 >> 172.21.15.32:0/3835163993 conn(0x7f9e3c071b90 msgr2=0x7f9e3c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 -- 172.21.15.32:0/3835163993 shutdown_connections 2023-11-24T07:59:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.451+0000 7f9e40a01700 1 -- 172.21.15.32:0/3835163993 wait complete. 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 Processor -- start 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 -- start start 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 0x7f9e3c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e3c1446a0 0x7f9e3c13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c15ac00 con 0x7f9e3c06c540 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c158340 con 0x7f9e3c0adf00 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e40a01700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c167f00 con 0x7f9e3c1446a0 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 0x7f9e3c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e3a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 0x7f9e3c15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58914/0 (socket says 172.21.15.32:58914) 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.453+0000 7f9e3a59c700 1 -- 172.21.15.32:0/1147381817 learned_addr learned my addr 172.21.15.32:0/1147381817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3ad9d700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e3c1446a0 0x7f9e3c13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e39d9b700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3a59c700 1 -- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e3c1446a0 msgr2=0x7f9e3c13bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3a59c700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e3c1446a0 0x7f9e3c13bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3a59c700 1 -- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c0adf00 msgr2=0x7f9e3c139f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3a59c700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c139f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3a59c700 1 -- 172.21.15.32:0/1147381817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c009550 con 0x7f9e3c06c540 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e3a59c700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 0x7f9e3c15ecb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000e810 tx=0x7f9e3000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000c020 con 0x7f9e3c06c540 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30004250 con 0x7f9e3c06c540 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c1681e0 con 0x7f9e3c06c540 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300036a0 con 0x7f9e3c06c540 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.454+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c158590 con 0x7f9e3c06c540 2023-11-24T07:59:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.455+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c0753f0 con 0x7f9e3c06c540 2023-11-24T07:59:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.456+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e30003800 con 0x7f9e3c06c540 2023-11-24T07:59:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.456+0000 7f9e2affd700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9e1c06af00 0x7f9e1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.456+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e30088ff0 con 0x7f9e3c06c540 2023-11-24T07:59:52.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.456+0000 7f9e39d9b700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9e1c06af00 0x7f9e1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.457+0000 7f9e39d9b700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9e1c06af00 0x7f9e1c06d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0729a0 tx=0x7f9e2c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.461+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e30054130 con 0x7f9e3c06c540 2023-11-24T07:59:52.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.704+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9e3c0ae2e0 con 0x7f9e3c06c540 2023-11-24T07:59:52.706 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-11-24T07:59:52.704+0000 7f61267f6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:59:52.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.706+0000 7f9e2affd700 1 -- 172.21.15.32:0/1147381817 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9e30057750 con 0x7f9e3c06c540 2023-11-24T07:59:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.708+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9e1c06af00 msgr2=0x7f9e1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.708+0000 7f9e40a01700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9e1c06af00 0x7f9e1c06d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0729a0 tx=0x7f9e2c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.708+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 msgr2=0x7f9e3c15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.708+0000 7f9e40a01700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 0x7f9e3c15ecb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000e810 tx=0x7f9e3000ebd0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.709+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 shutdown_connections 2023-11-24T07:59:52.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.709+0000 7f9e40a01700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f9e1c06af00 0x7f9e1c06d3c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.709+0000 7f9e40a01700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e3c1446a0 0x7f9e3c13bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.709+0000 7f9e40a01700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c06c540 0x7f9e3c15ecb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.709+0000 7f9e40a01700 1 --2- 172.21.15.32:0/1147381817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c0adf00 0x7f9e3c139f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.709+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 >> 172.21.15.32:0/1147381817 conn(0x7f9e3c071b90 msgr2=0x7f9e3c0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.710+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 shutdown_connections 2023-11-24T07:59:52.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.710+0000 7f9e40a01700 1 -- 172.21.15.32:0/1147381817 wait complete. 2023-11-24T07:59:52.714 INFO:tasks.ceph.mds.a.smithi032.stderr: -3185> 2023-11-24T07:59:38.670+0000 7f5b51656700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:59:52.714 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:59:52.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:59:52.742 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-11-24T07:59:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.851+0000 7f8bad883700 1 Processor -- start 2023-11-24T07:59:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.851+0000 7f8bad883700 1 -- start start 2023-11-24T07:59:52.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.852+0000 7f8bad883700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ba80b4370 0x7f8ba815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.852+0000 7f8bad883700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba80ae5e0 con 0x7f8ba81587f0 2023-11-24T07:59:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.852+0000 7f8bad883700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba815be30 con 0x7f8ba815b370 2023-11-24T07:59:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.852+0000 7f8bad883700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba815c110 con 0x7f8ba80b4370 2023-11-24T07:59:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.852+0000 7f8ba6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ba81587f0 0x7f8ba806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60646/0 (socket says 172.21.15.32:60646) 2023-11-24T07:59:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.852+0000 7f8ba6ffd700 1 -- 172.21.15.32:0/2045033408 learned_addr learned my addr 172.21.15.32:0/2045033408 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba77fe700 1 --2- 172.21.15.32:0/2045033408 >> v2:172.21.15.123:3300/0 conn(0x7f8ba80b4370 0x7f8ba815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba5ffb700 1 -- 172.21.15.32:0/2045033408 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402038179 0 0) 0x7f8ba80ae5e0 con 0x7f8ba81587f0 2023-11-24T07:59:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba5ffb700 1 -- 172.21.15.32:0/2045033408 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b900035b0 con 0x7f8ba81587f0 2023-11-24T07:59:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba5ffb700 1 -- 172.21.15.32:0/2045033408 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4170034934 0 0) 0x7f8b900035b0 con 0x7f8ba81587f0 2023-11-24T07:59:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba77fe700 1 -- 172.21.15.32:0/2045033408 >> v1:172.21.15.123:6789/0 conn(0x7f8ba815b370 legacy=0x7f8ba80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba77fe700 1 -- 172.21.15.32:0/2045033408 >> v1:172.21.15.32:6789/0 conn(0x7f8ba81587f0 legacy=0x7f8ba806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba77fe700 1 -- 172.21.15.32:0/2045033408 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba80ae860 con 0x7f8ba80b4370 2023-11-24T07:59:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba77fe700 1 --2- 172.21.15.32:0/2045033408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba80b4370 0x7f8ba815b800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009dc0 tx=0x7f8ba000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a462ab83024c52ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba5ffb700 1 -- 172.21.15.32:0/2045033408 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000e070 con 0x7f8ba80b4370 2023-11-24T07:59:52.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba5ffb700 1 -- 172.21.15.32:0/2045033408 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0003a20 con 0x7f8ba80b4370 2023-11-24T07:59:52.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.853+0000 7f8ba5ffb700 1 -- 172.21.15.32:0/2045033408 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c001670 con 0x7f8ba80b4370 2023-11-24T07:59:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 -- 172.21.15.32:0/2045033408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba80b4370 msgr2=0x7f8ba815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 --2- 172.21.15.32:0/2045033408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba80b4370 0x7f8ba815b800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009dc0 tx=0x7f8ba000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 -- 172.21.15.32:0/2045033408 shutdown_connections 2023-11-24T07:59:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 --2- 172.21.15.32:0/2045033408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba80b4370 0x7f8ba815b800 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 -- 172.21.15.32:0/2045033408 >> 172.21.15.32:0/2045033408 conn(0x7f8ba80d0bf0 msgr2=0x7f8ba80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 -- 172.21.15.32:0/2045033408 shutdown_connections 2023-11-24T07:59:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.854+0000 7f8bad883700 1 -- 172.21.15.32:0/2045033408 wait complete. 2023-11-24T07:59:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.855+0000 7f8bad883700 1 Processor -- start 2023-11-24T07:59:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.855+0000 7f8bad883700 1 -- start start 2023-11-24T07:59:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8bad883700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 0x7f8ba80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8bad883700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba81587f0 0x7f8ba8146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8bad883700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ba815b370 0x7f8ba814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8bad883700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8149c60 con 0x7f8ba80b4370 2023-11-24T07:59:52.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8bad883700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8158f60 con 0x7f8ba81587f0 2023-11-24T07:59:52.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8bad883700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80b0c30 con 0x7f8ba815b370 2023-11-24T07:59:52.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 0x7f8ba80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 0x7f8ba80b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58940/0 (socket says 172.21.15.32:58940) 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 -- 172.21.15.32:0/1755282293 learned_addr learned my addr 172.21.15.32:0/1755282293 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba77fe700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ba815b370 0x7f8ba814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba67fc700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba81587f0 0x7f8ba8146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 -- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ba815b370 msgr2=0x7f8ba814d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ba815b370 0x7f8ba814d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 -- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba81587f0 msgr2=0x7f8ba8146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba81587f0 0x7f8ba8146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba6ffd700 1 -- 172.21.15.32:0/1755282293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0009a70 con 0x7f8ba80b4370 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.856+0000 7f8ba67fc700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba81587f0 0x7f8ba8146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.857+0000 7f8ba6ffd700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 0x7f8ba80b0650 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c750 tx=0x7f8b9c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.857+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c004c80 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.857+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c004e00 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.857+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba8149ee0 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.857+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c007310 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.857+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba814a4a0 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.858+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba8147690 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.858+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b9c007470 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.859+0000 7f8b8ffff700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8b9406aed0 0x7f8b9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.859+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8b9c088b50 con 0x7f8ba80b4370 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.859+0000 7f8ba67fc700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8b9406aed0 0x7f8b9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.859+0000 7f8ba67fc700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8b9406aed0 0x7f8b9406d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8ba81f6f00 tx=0x7f8b98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:52.863+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b9c053ba0 con 0x7f8ba80b4370 2023-11-24T07:59:53.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.110+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ba8068fb0 con 0x7f8ba80b4370 2023-11-24T07:59:53.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.707+0000 7f8b8ffff700 1 -- 172.21.15.32:0/1755282293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b9c0180a0 con 0x7f8ba80b4370 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8b9406aed0 msgr2=0x7f8b9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8b9406aed0 0x7f8b9406d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8ba81f6f00 tx=0x7f8b98009040 comp rx=0 tx=0).stop 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 msgr2=0x7f8ba80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 0x7f8ba80b0650 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c750 tx=0x7f8b9c00cb10 comp rx=0 tx=0).stop 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 shutdown_connections 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8b9406aed0 0x7f8b9406d390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ba815b370 0x7f8ba814d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba80b4370 0x7f8ba80b0650 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 --2- 172.21.15.32:0/1755282293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba81587f0 0x7f8ba8146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 >> 172.21.15.32:0/1755282293 conn(0x7f8ba80d0bf0 msgr2=0x7f8ba806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:53.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 shutdown_connections 2023-11-24T07:59:53.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.710+0000 7f8bad883700 1 -- 172.21.15.32:0/1755282293 wait complete. 2023-11-24T07:59:53.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:59:53.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.903+0000 7fe38022e700 1 Processor -- start 2023-11-24T07:59:53.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.903+0000 7fe38022e700 1 -- start start 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe38022e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3780cf040 0x7fe37806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe38022e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3780d01c0 con 0x7fe3780d21c0 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe38022e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe378070910 con 0x7fe3780702e0 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe38022e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe378070bf0 con 0x7fe3780cf040 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37dfca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3780cf040 0x7fe37806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37e7cb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3780d21c0 0x7fe3780af3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60666/0 (socket says 172.21.15.32:60666) 2023-11-24T07:59:53.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37e7cb700 1 -- 172.21.15.32:0/2011123948 learned_addr learned my addr 172.21.15.32:0/2011123948 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37dfca700 1 -- 172.21.15.32:0/2011123948 >> v1:172.21.15.123:6789/0 conn(0x7fe3780702e0 legacy=0x7fe3780d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37dfca700 1 -- 172.21.15.32:0/2011123948 >> v1:172.21.15.32:6789/0 conn(0x7fe3780d21c0 legacy=0x7fe3780af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37dfca700 1 -- 172.21.15.32:0/2011123948 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3780aebb0 con 0x7fe3780cf040 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.904+0000 7fe37cfc8700 1 -- 172.21.15.32:0/2011123948 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458742570 0 0) 0x7fe3780d01c0 con 0x7fe3780d21c0 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe37dfca700 1 --2- 172.21.15.32:0/2011123948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe37806fda0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe374009840 tx=0x7fe374009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9528a9baec9e2599 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe37cfc8700 1 -- 172.21.15.32:0/2011123948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374004230 con 0x7fe3780cf040 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe37cfc8700 1 -- 172.21.15.32:0/2011123948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374004390 con 0x7fe3780cf040 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe37cfc8700 1 -- 172.21.15.32:0/2011123948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740036a0 con 0x7fe3780cf040 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 -- 172.21.15.32:0/2011123948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 msgr2=0x7fe37806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 --2- 172.21.15.32:0/2011123948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe37806fda0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe374009840 tx=0x7fe374009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 -- 172.21.15.32:0/2011123948 shutdown_connections 2023-11-24T07:59:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 --2- 172.21.15.32:0/2011123948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe37806fda0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 -- 172.21.15.32:0/2011123948 >> 172.21.15.32:0/2011123948 conn(0x7fe37809aa00 msgr2=0x7fe37809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 -- 172.21.15.32:0/2011123948 shutdown_connections 2023-11-24T07:59:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.905+0000 7fe38022e700 1 -- 172.21.15.32:0/2011123948 wait complete. 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.906+0000 7fe38022e700 1 Processor -- start 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 -- start start 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3780702e0 0x7fe3781f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe3780b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3780d21c0 0x7fe3780ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe378154c30 con 0x7fe3780cf040 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3780ce320 con 0x7fe3780d21c0 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe38022e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3780ce600 con 0x7fe3780702e0 2023-11-24T07:59:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe37d7c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe3780b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe37d7c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe3780b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58958/0 (socket says 172.21.15.32:58958) 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.907+0000 7fe37d7c9700 1 -- 172.21.15.32:0/1885289877 learned_addr learned my addr 172.21.15.32:0/1885289877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37dfca700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3780702e0 0x7fe3781f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37e7cb700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3780d21c0 0x7fe3780ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37d7c9700 1 -- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3780702e0 msgr2=0x7fe3781f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37d7c9700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3780702e0 0x7fe3781f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37d7c9700 1 -- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3780d21c0 msgr2=0x7fe3780ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37d7c9700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3780d21c0 0x7fe3780ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37d7c9700 1 -- 172.21.15.32:0/1885289877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740094a0 con 0x7fe3780cf040 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37dfca700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3780702e0 0x7fe3781f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37e7cb700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3780d21c0 0x7fe3780ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe37d7c9700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe3780b6af0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c4d0 tx=0x7fe36800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368011790 con 0x7fe3780cf040 2023-11-24T07:59:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3680118f0 con 0x7fe3780cf040 2023-11-24T07:59:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3780ce8e0 con 0x7fe3780cf040 2023-11-24T07:59:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368011ca0 con 0x7fe3780cf040 2023-11-24T07:59:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.908+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3780b65d0 con 0x7fe3780cf040 2023-11-24T07:59:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.909+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3780b6760 con 0x7fe3780cf040 2023-11-24T07:59:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.909+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe36801c460 con 0x7fe3780cf040 2023-11-24T07:59:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.910+0000 7fe36e7fc700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe35c06aed0 0x7fe35c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.910+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe368089480 con 0x7fe3780cf040 2023-11-24T07:59:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.910+0000 7fe37dfca700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe35c06aed0 0x7fe35c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.911+0000 7fe37dfca700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe35c06aed0 0x7fe35c06d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe374009b90 tx=0x7fe374008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:53.915+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe368054560 con 0x7fe3780cf040 2023-11-24T07:59:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.116+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe378067f60 con 0x7fe3780cf040 2023-11-24T07:59:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.774+0000 7fe36e7fc700 1 -- 172.21.15.32:0/1885289877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe368057b80 con 0x7fe3780cf040 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe35c06aed0 msgr2=0x7fe35c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe35c06aed0 0x7fe35c06d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe374009b90 tx=0x7fe374008030 comp rx=0 tx=0).stop 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 msgr2=0x7fe3780b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe3780b6af0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c4d0 tx=0x7fe36800c890 comp rx=0 tx=0).stop 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 shutdown_connections 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe35c06aed0 0x7fe35c06d390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3780d21c0 0x7fe3780ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3780702e0 0x7fe3781f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 --2- 172.21.15.32:0/1885289877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3780cf040 0x7fe3780b6af0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 >> 172.21.15.32:0/1885289877 conn(0x7fe37809aa00 msgr2=0x7fe3780b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 shutdown_connections 2023-11-24T07:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.779+0000 7fe38022e700 1 -- 172.21.15.32:0/1885289877 wait complete. 2023-11-24T07:59:54.782 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:59:54.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:59:54.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.936+0000 7f48512b2700 1 Processor -- start 2023-11-24T07:59:54.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.936+0000 7f48512b2700 1 -- start start 2023-11-24T07:59:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f48512b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f484c0b6040 0x7f484c168d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f48512b2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c0d1e60 con 0x7f484c15e3c0 2023-11-24T07:59:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f48512b2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c0d2140 con 0x7f484c15dfe0 2023-11-24T07:59:54.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f48512b2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c0d2ae0 con 0x7f484c0b6040 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f484b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f484c0b6040 0x7f484c168d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f484a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f484c15e3c0 0x7f484c0b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60678/0 (socket says 172.21.15.32:60678) 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f484a7fc700 1 -- 172.21.15.32:0/2914591673 learned_addr learned my addr 172.21.15.32:0/2914591673 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f484b7fe700 1 -- 172.21.15.32:0/2914591673 >> v1:172.21.15.123:6789/0 conn(0x7f484c15dfe0 legacy=0x7f484c0d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f484b7fe700 1 -- 172.21.15.32:0/2914591673 >> v1:172.21.15.32:6789/0 conn(0x7f484c15e3c0 legacy=0x7f484c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f484b7fe700 1 -- 172.21.15.32:0/2914591673 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0d2420 con 0x7f484c0b6040 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.937+0000 7f4849ffb700 1 -- 172.21.15.32:0/2914591673 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843566802 0 0) 0x7f484c0d1e60 con 0x7f484c15e3c0 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f484b7fe700 1 --2- 172.21.15.32:0/2914591673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b6040 0x7f484c168d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4844009d90 tx=0x7f484400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74630e765546932 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f4849ffb700 1 -- 172.21.15.32:0/2914591673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484400e070 con 0x7f484c0b6040 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f4849ffb700 1 -- 172.21.15.32:0/2914591673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484400bb90 con 0x7f484c0b6040 2023-11-24T07:59:54.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f4849ffb700 1 -- 172.21.15.32:0/2914591673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844003a20 con 0x7f484c0b6040 2023-11-24T07:59:54.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 -- 172.21.15.32:0/2914591673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b6040 msgr2=0x7f484c168d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 --2- 172.21.15.32:0/2914591673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b6040 0x7f484c168d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4844009d90 tx=0x7f484400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:54.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 -- 172.21.15.32:0/2914591673 shutdown_connections 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 --2- 172.21.15.32:0/2914591673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c0b6040 0x7f484c168d60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 -- 172.21.15.32:0/2914591673 >> 172.21.15.32:0/2914591673 conn(0x7f484c0d0b10 msgr2=0x7f484c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 -- 172.21.15.32:0/2914591673 shutdown_connections 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.938+0000 7f48512b2700 1 -- 172.21.15.32:0/2914591673 wait complete. 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 Processor -- start 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 -- start start 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484c0b6040 0x7f484c1f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484c15dfe0 0x7f484c0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 0x7f484c0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c15eb70 con 0x7f484c15e3c0 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c1f53a0 con 0x7f484c15dfe0 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f48512b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0cbb40 con 0x7f484c0b6040 2023-11-24T07:59:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.940+0000 7f484b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 0x7f484c0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 0x7f484c0caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58974/0 (socket says 172.21.15.32:58974) 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 -- 172.21.15.32:0/3902437179 learned_addr learned my addr 172.21.15.32:0/3902437179 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484a7fc700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484c15dfe0 0x7f484c0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484affd700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484c0b6040 0x7f484c1f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 -- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484c0b6040 msgr2=0x7f484c1f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484c0b6040 0x7f484c1f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 -- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484c15dfe0 msgr2=0x7f484c0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484c15dfe0 0x7f484c0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 -- 172.21.15.32:0/3902437179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4844009a70 con 0x7f484c15e3c0 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484affd700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484c0b6040 0x7f484c1f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484a7fc700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484c15dfe0 0x7f484c0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f484b7fe700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 0x7f484c0caf80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f484400b730 tx=0x7f4844004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844003a20 con 0x7f484c15e3c0 2023-11-24T07:59:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4844012860 con 0x7f484c15e3c0 2023-11-24T07:59:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c0cbe20 con 0x7f484c15e3c0 2023-11-24T07:59:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844012c10 con 0x7f484c15e3c0 2023-11-24T07:59:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.941+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c0ccdd0 con 0x7f484c15e3c0 2023-11-24T07:59:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.942+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c0ccf60 con 0x7f484c15e3c0 2023-11-24T07:59:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.943+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f484401d420 con 0x7f484c15e3c0 2023-11-24T07:59:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.943+0000 7f48337fe700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f483806af00 0x7f483806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.943+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f484408b710 con 0x7f484c15e3c0 2023-11-24T07:59:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.943+0000 7f484affd700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f483806af00 0x7f483806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.944+0000 7f484affd700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f483806af00 0x7f483806d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4840003f40 tx=0x7f4840009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:54.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:54.947+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4844056930 con 0x7f484c15e3c0 2023-11-24T07:59:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.148+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f484c0b42e0 con 0x7f484c15e3c0 2023-11-24T07:59:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.776+0000 7f48337fe700 1 -- 172.21.15.32:0/3902437179 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4844004930 con 0x7f484c15e3c0 2023-11-24T07:59:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f483806af00 msgr2=0x7f483806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f483806af00 0x7f483806d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4840003f40 tx=0x7f4840009040 comp rx=0 tx=0).stop 2023-11-24T07:59:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 msgr2=0x7f484c0caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 0x7f484c0caf80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f484400b730 tx=0x7f4844004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 shutdown_connections 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f483806af00 0x7f483806d3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f484c15e3c0 0x7f484c0caf80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f484c0b6040 0x7f484c1f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 --2- 172.21.15.32:0/3902437179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484c15dfe0 0x7f484c0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 >> 172.21.15.32:0/3902437179 conn(0x7f484c0d0b10 msgr2=0x7f484c15d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 shutdown_connections 2023-11-24T07:59:55.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.779+0000 7f48512b2700 1 -- 172.21.15.32:0/3902437179 wait complete. 2023-11-24T07:59:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-11-24T07:59:55.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T07:59:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.929+0000 7f6aadce5700 1 Processor -- start 2023-11-24T07:59:55.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.929+0000 7f6aadce5700 1 -- start start 2023-11-24T07:59:55.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aadce5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6aa81587f0 0x7f6aa806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aadce5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 0x7f6aa815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aadce5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa806b530 con 0x7f6aa80b3c60 2023-11-24T07:59:55.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aadce5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa80ae5e0 con 0x7f6aa81587f0 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aadce5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa815be30 con 0x7f6aa815b370 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aa7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6aa80b3c60 0x7f6aa80af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60694/0 (socket says 172.21.15.32:60694) 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aa7fff700 1 -- 172.21.15.32:0/3567866088 learned_addr learned my addr 172.21.15.32:0/3567866088 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132746790 0 0) 0x7f6aa806b530 con 0x7f6aa80b3c60 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aa6ffd700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 0x7f6aa815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/3567866088 >> v2:172.21.15.123:3300/0 conn(0x7f6aa81587f0 0x7f6aa806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.930+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a940035b0 con 0x7f6aa80b3c60 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643423530 0 0) 0x7f6a940035b0 con 0x7f6aa80b3c60 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 msgr2=0x7f6aa815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 0x7f6aa815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 >> v2:172.21.15.123:3300/0 conn(0x7f6aa81587f0 msgr2=0x7f6aa806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 --2- 172.21.15.32:0/3567866088 >> v2:172.21.15.123:3300/0 conn(0x7f6aa81587f0 0x7f6aa806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa815c110 con 0x7f6aa80b3c60 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f6aa00019f0 con 0x7f6aa80b3c60 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 >> v1:172.21.15.32:6789/0 conn(0x7f6aa80b3c60 legacy=0x7f6aa80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6a94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 0x7f6a94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a94004d90 0x7f6a94005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80ae5e0 con 0x7f6a94003a00 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa815be30 con 0x7f6a94004d90 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9400ba10 con 0x7f6aa815b370 2023-11-24T07:59:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6a94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa6ffd700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 0x7f6a94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa7fff700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a94004d90 0x7f6a94005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa77fe700 1 -- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 msgr2=0x7f6a94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa815b370 0x7f6a94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa77fe700 1 -- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a94004d90 msgr2=0x7f6a94005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a94004d90 0x7f6a94005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.931+0000 7f6aa77fe700 1 -- 172.21.15.32:0/3567866088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940035b0 con 0x7f6a94003a00 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6a94003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800bfc0 tx=0x7f6a94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1b0f8064c19913 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98010c30 con 0x7f6a94003a00 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98011040 con 0x7f6a94003a00 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aa67fc700 1 -- 172.21.15.32:0/3567866088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98016480 con 0x7f6a94003a00 2023-11-24T07:59:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aadce5700 1 -- 172.21.15.32:0/3567866088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 msgr2=0x7f6a94003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aadce5700 1 --2- 172.21.15.32:0/3567866088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6a94003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800bfc0 tx=0x7f6a94005880 comp rx=0 tx=0).stop 2023-11-24T08:00:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aa7fff700 1 -- 172.21.15.32:0/3567866088 reap_dead start 2023-11-24T08:00:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aadce5700 1 -- 172.21.15.32:0/3567866088 shutdown_connections 2023-11-24T08:00:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aadce5700 1 -- 172.21.15.32:0/3567866088 >> 172.21.15.32:0/3567866088 conn(0x7f6aa80d0bf0 msgr2=0x7f6aa80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aadce5700 1 -- 172.21.15.32:0/3567866088 shutdown_connections 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.932+0000 7f6aadce5700 1 -- 172.21.15.32:0/3567866088 wait complete. 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 Processor -- start 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 -- start start 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6aa814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8152870 0x7f6aa8153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa81558b0 0x7f6aa814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80afd10 con 0x7f6a94003a00 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80af090 con 0x7f6aa8152870 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.934+0000 7f6aadce5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80af370 con 0x7f6aa81558b0 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6aa814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6aa814f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58990/0 (socket says 172.21.15.32:58990) 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 -- 172.21.15.32:0/4042181895 learned_addr learned my addr 172.21.15.32:0/4042181895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:04.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 -- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa81558b0 msgr2=0x7f6aa814e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa81558b0 0x7f6aa814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 -- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8152870 msgr2=0x7f6aa8153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8152870 0x7f6aa8153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 -- 172.21.15.32:0/4042181895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80af650 con 0x7f6a94003a00 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aa77fe700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6aa814f450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6a98010700 tx=0x7f6a9800b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98009000 con 0x7f6a94003a00 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98003ad0 con 0x7f6a94003a00 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80af930 con 0x7f6a94003a00 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003e80 con 0x7f6a94003a00 2023-11-24T08:00:04.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.935+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa8155d40 con 0x7f6a94003a00 2023-11-24T08:00:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.936+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa8155f60 con 0x7f6a94003a00 2023-11-24T08:00:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.937+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a98016480 con 0x7f6a94003a00 2023-11-24T08:00:05.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.937+0000 7f6a9ffff700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6a8806aed0 0x7f6a8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.937+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6a9808cf20 con 0x7f6a94003a00 2023-11-24T08:00:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.937+0000 7f6aa6ffd700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6a8806aed0 0x7f6a8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.938+0000 7f6aa6ffd700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6a8806aed0 0x7f6a8806d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81f28c0 tx=0x7f6a90009670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:55.942+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a98058360 con 0x7f6a94003a00 2023-11-24T08:00:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.142+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6aa8068fb0 con 0x7f6a94003a00 2023-11-24T08:00:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.146+0000 7f6a9ffff700 1 -- 172.21.15.32:0/4042181895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a9805b980 con 0x7f6a94003a00 2023-11-24T08:00:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6a8806aed0 msgr2=0x7f6a8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6a8806aed0 0x7f6a8806d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81f28c0 tx=0x7f6a90009670 comp rx=0 tx=0).stop 2023-11-24T08:00:05.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 msgr2=0x7f6aa814f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6aa814f450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6a98010700 tx=0x7f6a9800b750 comp rx=0 tx=0).stop 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 shutdown_connections 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6a8806aed0 0x7f6a8806d390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa81558b0 0x7f6aa814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a94003a00 0x7f6aa814f450 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 --2- 172.21.15.32:0/4042181895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8152870 0x7f6aa8153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 >> 172.21.15.32:0/4042181895 conn(0x7f6aa80d0bf0 msgr2=0x7f6aa8156880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 shutdown_connections 2023-11-24T08:00:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T07:59:56.148+0000 7f6aadce5700 1 -- 172.21.15.32:0/4042181895 wait complete. 2023-11-24T08:00:05.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2023-11-24T08:00:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.162+0000 7f950c93a700 1 Processor -- start 2023-11-24T08:00:05.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.162+0000 7f950c93a700 1 -- start start 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950c93a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9508145760 0x7f950806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950c93a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f950815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950c93a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f950815fde0 0x7f9508160250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950c93a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950806ee70 con 0x7f950815fde0 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950c93a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950806d4d0 con 0x7f9508145760 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950c93a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080cc5e0 con 0x7f950806df20 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f950815fde0 0x7f9508160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f950815fde0 0x7f9508160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59002/0 (socket says 172.21.15.32:59002) 2023-11-24T08:00:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 -- 172.21.15.32:0/2925503798 learned_addr learned my addr 172.21.15.32:0/2925503798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9505d9b700 1 --2- 172.21.15.32:0/2925503798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f950815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 -- 172.21.15.32:0/2925503798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 msgr2=0x7f950815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 --2- 172.21.15.32:0/2925503798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f950815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950659c700 1 --2- 172.21.15.32:0/2925503798 >> v2:172.21.15.123:3300/0 conn(0x7f9508145760 0x7f950806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 -- 172.21.15.32:0/2925503798 >> v2:172.21.15.123:3300/0 conn(0x7f9508145760 msgr2=0x7f950806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 --2- 172.21.15.32:0/2925503798 >> v2:172.21.15.123:3300/0 conn(0x7f9508145760 0x7f950806e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 -- 172.21.15.32:0/2925503798 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95080cc8c0 con 0x7f950815fde0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f9506d9d700 1 --2- 172.21.15.32:0/2925503798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508160250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f950804fca0 tx=0x7f950000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9bb50990aa09a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.163+0000 7f950559a700 1 -- 172.21.15.32:0/2925503798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000e070 con 0x7f950815fde0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950559a700 1 -- 172.21.15.32:0/2925503798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950000b9c0 con 0x7f950815fde0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950559a700 1 -- 172.21.15.32:0/2925503798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000bd70 con 0x7f950815fde0 2023-11-24T08:00:05.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 -- 172.21.15.32:0/2925503798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 msgr2=0x7f9508160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 --2- 172.21.15.32:0/2925503798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508160250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f950804fca0 tx=0x7f950000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 -- 172.21.15.32:0/2925503798 shutdown_connections 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 --2- 172.21.15.32:0/2925503798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508160250 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 --2- 172.21.15.32:0/2925503798 >> v2:172.21.15.123:3300/0 conn(0x7f9508145760 0x7f950806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 --2- 172.21.15.32:0/2925503798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f950815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 -- 172.21.15.32:0/2925503798 >> 172.21.15.32:0/2925503798 conn(0x7f95080b1740 msgr2=0x7f95080b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 -- 172.21.15.32:0/2925503798 shutdown_connections 2023-11-24T08:00:05.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.164+0000 7f950c93a700 1 -- 172.21.15.32:0/2925503798 wait complete. 2023-11-24T08:00:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 Processor -- start 2023-11-24T08:00:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 -- start start 2023-11-24T08:00:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f95080aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508145760 0x7f950813d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b4aa0 con 0x7f950815fde0 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b3e90 con 0x7f9508145760 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.166+0000 7f950c93a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b4170 con 0x7f950806df20 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59008/0 (socket says 172.21.15.32:59008) 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 -- 172.21.15.32:0/3444064349 learned_addr learned my addr 172.21.15.32:0/3444064349 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f950659c700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f95080aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9505d9b700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508145760 0x7f950813d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 -- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 msgr2=0x7f95080aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f95080aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 -- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508145760 msgr2=0x7f950813d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508145760 0x7f950813d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 -- 172.21.15.32:0/3444064349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500009b30 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f9506d9d700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508152e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f950000b370 tx=0x7f9500004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500004250 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95000043f0 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95080b4450 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95000047a0 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.167+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95080af0c0 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.168+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950815c980 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.169+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f950001b430 con 0x7f950815fde0 2023-11-24T08:00:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.169+0000 7f94f6ffd700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94e806aed0 0x7f94e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.169+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f950008a4c0 con 0x7f950815fde0 2023-11-24T08:00:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.170+0000 7f950659c700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94e806aed0 0x7f94e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.170+0000 7f950659c700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94e806aed0 0x7f94e806d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f94f800ab60 tx=0x7f94f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.174+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9500055870 con 0x7f950815fde0 2023-11-24T08:00:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.458+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7f95080751e0 con 0x7f950815fde0 2023-11-24T08:00:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.805+0000 7f94f6ffd700 1 -- 172.21.15.32:0/3444064349 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v99) v1 ==== 167+0+0 (secure 0 0 0) 0x7f950001f0e0 con 0x7f950815fde0 2023-11-24T08:00:05.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94e806aed0 msgr2=0x7f94e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94e806aed0 0x7f94e806d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f94f800ab60 tx=0x7f94f8008040 comp rx=0 tx=0).stop 2023-11-24T08:00:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 msgr2=0x7f9508152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508152e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f950000b370 tx=0x7f9500004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 shutdown_connections 2023-11-24T08:00:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f94e806aed0 0x7f94e806d390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950815fde0 0x7f9508152e40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f950806df20 0x7f95080aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 --2- 172.21.15.32:0/3444064349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508145760 0x7f950813d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 >> 172.21.15.32:0/3444064349 conn(0x7f95080b1740 msgr2=0x7f95080ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 shutdown_connections 2023-11-24T08:00:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:05.808+0000 7f950c93a700 1 -- 172.21.15.32:0/3444064349 wait complete. 2023-11-24T08:00:05.825 INFO:tasks.cephfs_test_runner:test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) ... ERROR 2023-11-24T08:00:05.825 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:05.825 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner:ERROR: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 8014, in test_volumes_module_finisher_thread 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner: subvol1, subvol2, subvol3 = self._generate_random_subvolume_name(3) 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner:AttributeError: 'TestPerModuleFinsherThread' object has no attribute '_generate_random_subvolume_name' 2023-11-24T08:00:05.826 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner:Ran 1 test in 57.805s 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:00:05.827 INFO:tasks.cephfs_test_runner:ERROR: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T08:00:05.828 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:00:05.828 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:00:05.828 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 8014, in test_volumes_module_finisher_thread 2023-11-24T08:00:05.828 INFO:tasks.cephfs_test_runner: subvol1, subvol2, subvol3 = self._generate_random_subvolume_name(3) 2023-11-24T08:00:05.828 INFO:tasks.cephfs_test_runner:AttributeError: 'TestPerModuleFinsherThread' object has no attribute '_generate_random_subvolume_name' 2023-11-24T08:00:05.828 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:05.828 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T08:00:05.901 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=906c694f5a854172af02d98e4268a9f9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-11-24T08:00:05.902 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T08:00:05.915 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-24T08:00:05.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.056+0000 7f94bcaed700 1 Processor -- start 2023-11-24T08:00:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.056+0000 7f94bcaed700 1 -- start start 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94bcaed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94b815bd60 0x7f94b80d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94bcaed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94b815c140 0x7f94b80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94bcaed700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b80cc780 con 0x7f94b80cf2b0 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94bcaed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b80cc9b0 con 0x7f94b815c140 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94bcaed700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b80ccc90 con 0x7f94b815bd60 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94b5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94b815c140 0x7f94b80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94b5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94b815c140 0x7f94b80cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59012/0 (socket says 172.21.15.32:59012) 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.057+0000 7f94b5d9b700 1 -- 172.21.15.32:0/1962140487 learned_addr learned my addr 172.21.15.32:0/1962140487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b659c700 1 --2- 172.21.15.32:0/1962140487 >> v2:172.21.15.123:3300/0 conn(0x7f94b815bd60 0x7f94b80d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b5d9b700 1 -- 172.21.15.32:0/1962140487 >> v2:172.21.15.123:3300/0 conn(0x7f94b815bd60 msgr2=0x7f94b80d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b5d9b700 1 --2- 172.21.15.32:0/1962140487 >> v2:172.21.15.123:3300/0 conn(0x7f94b815bd60 0x7f94b80d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b5d9b700 1 -- 172.21.15.32:0/1962140487 >> v1:172.21.15.123:6789/0 conn(0x7f94b80cf2b0 legacy=0x7f94b80c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b5d9b700 1 -- 172.21.15.32:0/1962140487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b80ccf70 con 0x7f94b815c140 2023-11-24T08:00:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b659c700 1 --2- 172.21.15.32:0/1962140487 >> v2:172.21.15.123:3300/0 conn(0x7f94b815bd60 0x7f94b80d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b5d9b700 1 --2- 172.21.15.32:0/1962140487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b815c140 0x7f94b80cda50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f94b80c8cf0 tx=0x7f94a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4711415460e7b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b559a700 1 -- 172.21.15.32:0/1962140487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a800b870 con 0x7f94b815c140 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b559a700 1 -- 172.21.15.32:0/1962140487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a800b9d0 con 0x7f94b815c140 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.058+0000 7f94b559a700 1 -- 172.21.15.32:0/1962140487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a800bd60 con 0x7f94b815c140 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 -- 172.21.15.32:0/1962140487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b815c140 msgr2=0x7f94b80cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 --2- 172.21.15.32:0/1962140487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b815c140 0x7f94b80cda50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f94b80c8cf0 tx=0x7f94a8009bd0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 -- 172.21.15.32:0/1962140487 shutdown_connections 2023-11-24T08:00:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 --2- 172.21.15.32:0/1962140487 >> v2:172.21.15.123:3300/0 conn(0x7f94b815bd60 0x7f94b80d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 --2- 172.21.15.32:0/1962140487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b815c140 0x7f94b80cda50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 -- 172.21.15.32:0/1962140487 >> 172.21.15.32:0/1962140487 conn(0x7f94b8073130 msgr2=0x7f94b809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 -- 172.21.15.32:0/1962140487 shutdown_connections 2023-11-24T08:00:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.059+0000 7f94bcaed700 1 -- 172.21.15.32:0/1962140487 wait complete. 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.060+0000 7f94bcaed700 1 Processor -- start 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.060+0000 7f94bcaed700 1 -- start start 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94bcaed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 0x7f94b81f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94bcaed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b815bd60 0x7f94b80b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94bcaed700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94b815c140 0x7f94b80b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94bcaed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b5d70 con 0x7f94b80cf2b0 2023-11-24T08:00:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94bcaed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81f1840 con 0x7f94b815bd60 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94bcaed700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81f1b20 con 0x7f94b815c140 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 0x7f94b81f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 0x7f94b81f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59016/0 (socket says 172.21.15.32:59016) 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 -- 172.21.15.32:0/3893642815 learned_addr learned my addr 172.21.15.32:0/3893642815 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b5d9b700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b815bd60 0x7f94b80b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b6d9d700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94b815c140 0x7f94b80b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 -- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94b815c140 msgr2=0x7f94b80b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94b815c140 0x7f94b80b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 -- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b815bd60 msgr2=0x7f94b80b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b815bd60 0x7f94b80b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 -- 172.21.15.32:0/3893642815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8009550 con 0x7f94b80cf2b0 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.061+0000 7f94b659c700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 0x7f94b81f0e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e810 tx=0x7f94a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.062+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0013610 con 0x7f94b80cf2b0 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.062+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a0013770 con 0x7f94b80cf2b0 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.062+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b81f1e00 con 0x7f94b80cf2b0 2023-11-24T08:00:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.062+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0013b20 con 0x7f94b80cf2b0 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.062+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b81f23c0 con 0x7f94b80cf2b0 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.063+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b8067f60 con 0x7f94b80cf2b0 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.063+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94a0013c80 con 0x7f94b80cf2b0 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.064+0000 7f94aeffd700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f949806aed0 0x7f949806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.064+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f94a0088e20 con 0x7f94b80cf2b0 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.064+0000 7f94b5d9b700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f949806aed0 0x7f949806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.064+0000 7f94b5d9b700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f949806aed0 0x7f949806d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f94a8014040 tx=0x7f94a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.068+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94a00541d0 con 0x7f94b80cf2b0 2023-11-24T08:00:06.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.270+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94b80b7bf0 con 0x7f94b80cf2b0 2023-11-24T08:00:06.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.271+0000 7f94aeffd700 1 -- 172.21.15.32:0/3893642815 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+12972 (secure 0 0 0) 0x7f94a00577f0 con 0x7f94b80cf2b0 2023-11-24T08:00:06.273 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:06.273 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:55.775681+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/598023464":"2023-11-25T07:59:52.705443+0000","172.21.15.32:6833/598023464":"2023-11-25T07:59:52.705443+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:06.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f949806aed0 msgr2=0x7f949806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f949806aed0 0x7f949806d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f94a8014040 tx=0x7f94a8008030 comp rx=0 tx=0).stop 2023-11-24T08:00:06.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 msgr2=0x7f94b81f0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 0x7f94b81f0e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e810 tx=0x7f94a000eb20 comp rx=0 tx=0).stop 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 shutdown_connections 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f949806aed0 0x7f949806d390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94b815c140 0x7f94b80b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf2b0 0x7f94b81f0e40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 --2- 172.21.15.32:0/3893642815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b815bd60 0x7f94b80b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 >> 172.21.15.32:0/3893642815 conn(0x7f94b8073130 msgr2=0x7f94b809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 shutdown_connections 2023-11-24T08:00:06.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.273+0000 7f94bcaed700 1 -- 172.21.15.32:0/3893642815 wait complete. 2023-11-24T08:00:06.287 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:58:32.587117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:00:06.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:00:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.414+0000 7f9307867700 1 Processor -- start 2023-11-24T08:00:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.414+0000 7f9307867700 1 -- start start 2023-11-24T08:00:06.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9307867700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93000b4020 0x7f9300073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9307867700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f93000744d0 0x7f930006d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9307867700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930015bf70 con 0x7f93000af520 2023-11-24T08:00:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9307867700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9300074960 con 0x7f93000b4020 2023-11-24T08:00:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9307867700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930015ee00 con 0x7f93000744d0 2023-11-24T08:00:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9305e04700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f93000744d0 0x7f930006d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9304e02700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93000b4020 0x7f9300073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9305603700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93000af520 0x7f93000afb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60736/0 (socket says 172.21.15.32:60736) 2023-11-24T08:00:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.415+0000 7f9305603700 1 -- 172.21.15.32:0/3081878922 learned_addr learned my addr 172.21.15.32:0/3081878922 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f9304e02700 1 -- 172.21.15.32:0/3081878922 >> v2:172.21.15.123:3300/0 conn(0x7f93000744d0 msgr2=0x7f930006d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f9304e02700 1 --2- 172.21.15.32:0/3081878922 >> v2:172.21.15.123:3300/0 conn(0x7f93000744d0 0x7f930006d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f9304e02700 1 -- 172.21.15.32:0/3081878922 >> v1:172.21.15.32:6789/0 conn(0x7f93000af520 legacy=0x7f93000afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f9304e02700 1 -- 172.21.15.32:0/3081878922 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000b0240 con 0x7f93000b4020 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f92f7fff700 1 -- 172.21.15.32:0/3081878922 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455462269 0 0) 0x7f92fc001110 con 0x7f93000af520 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f9304e02700 1 --2- 172.21.15.32:0/3081878922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000b4020 0x7f9300073f90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9300072c90 tx=0x7f92f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fbb913087893c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f92f7fff700 1 -- 172.21.15.32:0/3081878922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0011830 con 0x7f93000b4020 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f92f7fff700 1 -- 172.21.15.32:0/3081878922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f0011990 con 0x7f93000b4020 2023-11-24T08:00:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.416+0000 7f92f7fff700 1 -- 172.21.15.32:0/3081878922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0011d20 con 0x7f93000b4020 2023-11-24T08:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 -- 172.21.15.32:0/3081878922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000b4020 msgr2=0x7f9300073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 --2- 172.21.15.32:0/3081878922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000b4020 0x7f9300073f90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9300072c90 tx=0x7f92f0009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 -- 172.21.15.32:0/3081878922 shutdown_connections 2023-11-24T08:00:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 --2- 172.21.15.32:0/3081878922 >> v2:172.21.15.123:3300/0 conn(0x7f93000744d0 0x7f930006d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 --2- 172.21.15.32:0/3081878922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000b4020 0x7f9300073f90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 -- 172.21.15.32:0/3081878922 >> 172.21.15.32:0/3081878922 conn(0x7f93000b28f0 msgr2=0x7f93000b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 -- 172.21.15.32:0/3081878922 shutdown_connections 2023-11-24T08:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.417+0000 7f9307867700 1 -- 172.21.15.32:0/3081878922 wait complete. 2023-11-24T08:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.418+0000 7f9307867700 1 Processor -- start 2023-11-24T08:00:06.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.418+0000 7f9307867700 1 -- start start 2023-11-24T08:00:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9307867700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93000744d0 0x7f9300146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9307867700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 0x7f930006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9307867700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93000b4020 0x7f930006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9307867700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930015f290 con 0x7f93000af520 2023-11-24T08:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9307867700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930015d160 con 0x7f93000744d0 2023-11-24T08:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9307867700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930015d440 con 0x7f93000b4020 2023-11-24T08:00:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 0x7f930006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 0x7f930006ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59034/0 (socket says 172.21.15.32:59034) 2023-11-24T08:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 -- 172.21.15.32:0/971295130 learned_addr learned my addr 172.21.15.32:0/971295130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9305603700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93000744d0 0x7f9300146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9305e04700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93000b4020 0x7f930006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 -- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93000b4020 msgr2=0x7f930006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93000b4020 0x7f930006bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 -- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93000744d0 msgr2=0x7f9300146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93000744d0 0x7f9300146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 -- 172.21.15.32:0/971295130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00094a0 con 0x7f93000af520 2023-11-24T08:00:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9305e04700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93000b4020 0x7f930006bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.419+0000 7f9304e02700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 0x7f930006ba00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f92f0014040 tx=0x7f92f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.420+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f00036a0 con 0x7f93000af520 2023-11-24T08:00:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.420+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f001f070 con 0x7f93000af520 2023-11-24T08:00:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.420+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930015f550 con 0x7f93000af520 2023-11-24T08:00:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.420+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0003a20 con 0x7f93000af520 2023-11-24T08:00:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.420+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9300146880 con 0x7f93000af520 2023-11-24T08:00:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.421+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9300068fb0 con 0x7f93000af520 2023-11-24T08:00:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.421+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92f0003b80 con 0x7f93000af520 2023-11-24T08:00:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.422+0000 7f92f5ffb700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f92e406aed0 0x7f92e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.422+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f92f0021070 con 0x7f93000af520 2023-11-24T08:00:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.422+0000 7f9305603700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f92e406aed0 0x7f92e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.422+0000 7f9305603700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f92e406aed0 0x7f92e406d390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f92fc000f60 tx=0x7f92fc007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.426+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92f0055dd0 con 0x7f93000af520 2023-11-24T08:00:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.628+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f930006b4e0 con 0x7f93000af520 2023-11-24T08:00:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.628+0000 7f92f5ffb700 1 -- 172.21.15.32:0/971295130 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92f0015020 con 0x7f93000af520 2023-11-24T08:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-11-24T08:00:06.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f92e406aed0 msgr2=0x7f92e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f92e406aed0 0x7f92e406d390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f92fc000f60 tx=0x7f92fc007c20 comp rx=0 tx=0).stop 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 msgr2=0x7f930006ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 0x7f930006ba00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f92f0014040 tx=0x7f92f0004750 comp rx=0 tx=0).stop 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 shutdown_connections 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f92e406aed0 0x7f92e406d390 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f93000b4020 0x7f930006bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93000744d0 0x7f9300146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 --2- 172.21.15.32:0/971295130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93000af520 0x7f930006ba00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.630+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 >> 172.21.15.32:0/971295130 conn(0x7f93000b28f0 msgr2=0x7f93000add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.631+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 shutdown_connections 2023-11-24T08:00:06.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.631+0000 7f9307867700 1 -- 172.21.15.32:0/971295130 wait complete. 2023-11-24T08:00:06.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-24T08:00:06.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.770+0000 7f008e698700 1 Processor -- start 2023-11-24T08:00:06.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.770+0000 7f008e698700 1 -- start start 2023-11-24T08:00:06.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.770+0000 7f008e698700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00880ae910 0x7f00880ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.770+0000 7f008e698700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00880cd5d0 con 0x7f008815c030 2023-11-24T08:00:06.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.770+0000 7f008e698700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00880cd870 con 0x7f00880d0780 2023-11-24T08:00:06.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.770+0000 7f008e698700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00880cdb50 con 0x7f00880ae910 2023-11-24T08:00:06.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f008cc35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00880ae910 0x7f00880ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f0087fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f008815c030 0x7f00880d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60750/0 (socket says 172.21.15.32:60750) 2023-11-24T08:00:06.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f0087fff700 1 -- 172.21.15.32:0/3896480130 learned_addr learned my addr 172.21.15.32:0/3896480130 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:06.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f008cc35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00880ae910 0x7f00880ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59040/0 (socket says 172.21.15.32:59040) 2023-11-24T08:00:06.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f008cc35700 1 -- 172.21.15.32:0/3896480130 >> v1:172.21.15.123:6789/0 conn(0x7f00880d0780 legacy=0x7f00880cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f008cc35700 1 -- 172.21.15.32:0/3896480130 >> v1:172.21.15.32:6789/0 conn(0x7f008815c030 legacy=0x7f00880d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f0086ffd700 1 -- 172.21.15.32:0/3896480130 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846756642 0 0) 0x7f00880cd5d0 con 0x7f008815c030 2023-11-24T08:00:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f008cc35700 1 -- 172.21.15.32:0/3896480130 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880cde30 con 0x7f00880ae910 2023-11-24T08:00:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f008cc35700 1 --2- 172.21.15.32:0/3896480130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f00880ccfd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0080009d90 tx=0x7f008000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b26685ba0a48cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f0086ffd700 1 -- 172.21.15.32:0/3896480130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000e070 con 0x7f00880ae910 2023-11-24T08:00:06.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f0086ffd700 1 -- 172.21.15.32:0/3896480130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008000bb90 con 0x7f00880ae910 2023-11-24T08:00:06.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.771+0000 7f0086ffd700 1 -- 172.21.15.32:0/3896480130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080003a20 con 0x7f00880ae910 2023-11-24T08:00:06.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 -- 172.21.15.32:0/3896480130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 msgr2=0x7f00880ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 --2- 172.21.15.32:0/3896480130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f00880ccfd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0080009d90 tx=0x7f008000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:06.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 -- 172.21.15.32:0/3896480130 shutdown_connections 2023-11-24T08:00:06.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 --2- 172.21.15.32:0/3896480130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f00880ccfd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 -- 172.21.15.32:0/3896480130 >> 172.21.15.32:0/3896480130 conn(0x7f00880730a0 msgr2=0x7f008809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 -- 172.21.15.32:0/3896480130 shutdown_connections 2023-11-24T08:00:06.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.772+0000 7f008e698700 1 -- 172.21.15.32:0/3896480130 wait complete. 2023-11-24T08:00:06.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.773+0000 7f008e698700 1 Processor -- start 2023-11-24T08:00:06.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 -- start start 2023-11-24T08:00:06.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f0088152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00880d0780 0x7f008815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088152030 0x7f00880cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008815ab30 con 0x7f00880ae910 2023-11-24T08:00:06.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880b6010 con 0x7f0088152030 2023-11-24T08:00:06.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f008e698700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00880b5a50 con 0x7f00880d0780 2023-11-24T08:00:06.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f0087fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f0088152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f0087fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f0088152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59042/0 (socket says 172.21.15.32:59042) 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.774+0000 7f0087fff700 1 -- 172.21.15.32:0/2921524895 learned_addr learned my addr 172.21.15.32:0/2921524895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f00877fe700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00880d0780 0x7f008815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f008cc35700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088152030 0x7f00880cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0087fff700 1 -- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00880d0780 msgr2=0x7f008815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0087fff700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00880d0780 0x7f008815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0087fff700 1 -- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088152030 msgr2=0x7f00880cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0087fff700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088152030 0x7f00880cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0087fff700 1 -- 172.21.15.32:0/2921524895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080009a70 con 0x7f00880ae910 2023-11-24T08:00:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0087fff700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f0088152870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f007800c550 tx=0x7f007800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078004c80 con 0x7f00880ae910 2023-11-24T08:00:06.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078004e00 con 0x7f00880ae910 2023-11-24T08:00:06.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00880b5d30 con 0x7f00880ae910 2023-11-24T08:00:06.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078007310 con 0x7f00880ae910 2023-11-24T08:00:06.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.775+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00881524e0 con 0x7f00880ae910 2023-11-24T08:00:06.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.776+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00880751e0 con 0x7f00880ae910 2023-11-24T08:00:06.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.777+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0078007470 con 0x7f00880ae910 2023-11-24T08:00:06.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.777+0000 7f0084ff9700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f007406aed0 0x7f007406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.777+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0078089040 con 0x7f00880ae910 2023-11-24T08:00:06.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.777+0000 7f00877fe700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f007406aed0 0x7f007406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.778+0000 7f00877fe700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f007406aed0 0x7f007406d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f00881f7120 tx=0x7f007c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:06.781+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00780543f0 con 0x7f00880ae910 2023-11-24T08:00:07.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.046+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f008806b2d0 con 0x7f00880ae910 2023-11-24T08:00:07.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.051+0000 7f0084ff9700 1 -- 172.21.15.32:0/2921524895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v5) v1 ==== 81+0+466154 (secure 0 0 0) 0x7f008806b2d0 con 0x7f00880ae910 2023-11-24T08:00:07.058 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:07.058 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:07.058 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 5, 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "active_gid": 4105, 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "active_name": "x", 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addrs": { 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.123:6832", 2023-11-24T08:00:07.059 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1302929475 2023-11-24T08:00:07.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-11-24T08:00:07.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.123:6833", 2023-11-24T08:00:07.060 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1302929475 2023-11-24T08:00:07.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addr": "172.21.15.123:6833/1302929475", 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: "active_change": "2023-11-24T07:58:26.521793+0000", 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: "active_mgr_features": 4540138322906710015, 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: "available": true, 2023-11-24T08:00:07.061 INFO:teuthology.orchestra.run.smithi032.stdout: "standbys": [ 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 4106, 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "y", 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540138322906710015, 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.064 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.066 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.067 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.070 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.071 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.074 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.075 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.079 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-11-24T08:00:07.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-11-24T08:00:07.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-11-24T08:00:07.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-11-24T08:00:07.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-24T08:00:07.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.095 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.096 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.097 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.098 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-11-24T08:00:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.105 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.106 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.108 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.108 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.108 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.108 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.108 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.108 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.110 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.114 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-11-24T08:00:07.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.118 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:07.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-11-24T08:00:07.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.126 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-24T08:00:07.127 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.132 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2023-11-24T08:00:07.137 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.138 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.139 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.143 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.144 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-11-24T08:00:07.145 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.146 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-11-24T08:00:07.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-11-24T08:00:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-24T08:00:07.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.154 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.154 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.154 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-11-24T08:00:07.154 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-24T08:00:07.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.158 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-11-24T08:00:07.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-11-24T08:00:07.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.163 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-24T08:00:07.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.171 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.172 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-11-24T08:00:07.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.174 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.176 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.176 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.176 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.177 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-24T08:00:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.184 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-11-24T08:00:07.186 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.187 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2023-11-24T08:00:07.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-24T08:00:07.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.190 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.191 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.195 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.196 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.198 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-24T08:00:07.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-24T08:00:07.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.207 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-11-24T08:00:07.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-11-24T08:00:07.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.207 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.209 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.209 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.209 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.209 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-11-24T08:00:07.211 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.213 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-11-24T08:00:07.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-24T08:00:07.215 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.221 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.222 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-11-24T08:00:07.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-11-24T08:00:07.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.226 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.230 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.231 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.234 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.235 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-11-24T08:00:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-24T08:00:07.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.240 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.241 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.244 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.245 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.247 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-11-24T08:00:07.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.249 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-11-24T08:00:07.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-24T08:00:07.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.256 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-11-24T08:00:07.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-24T08:00:07.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.264 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.265 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.266 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.268 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-24T08:00:07.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-24T08:00:07.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.272 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.273 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.274 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-11-24T08:00:07.279 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.280 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-11-24T08:00:07.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.289 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-24T08:00:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.293 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.298 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.299 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.301 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-24T08:00:07.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-24T08:00:07.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.311 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-24T08:00:07.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-24T08:00:07.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.316 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.317 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.324 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.325 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2023-11-24T08:00:07.326 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.327 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.331 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:07.332 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-11-24T08:00:07.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.337 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-11-24T08:00:07.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.344 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.346 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.347 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.349 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.352 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.355 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-11-24T08:00:07.356 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.357 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-11-24T08:00:07.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-11-24T08:00:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.362 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-11-24T08:00:07.364 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-11-24T08:00:07.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.366 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.367 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.368 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-11-24T08:00:07.369 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.372 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.373 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.376 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.377 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.378 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.380 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-11-24T08:00:07.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-11-24T08:00:07.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-11-24T08:00:07.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.389 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.390 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.391 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-11-24T08:00:07.392 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.393 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.399 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.401 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.403 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.405 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.406 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.407 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:07.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-11-24T08:00:07.410 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-11-24T08:00:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-11-24T08:00:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.413 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:07.416 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.417 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.418 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.419 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.423 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.424 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.426 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.427 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.428 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-11-24T08:00:07.429 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.430 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-11-24T08:00:07.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-11-24T08:00:07.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-24T08:00:07.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.438 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.439 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.440 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.446 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.450 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.452 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.454 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.455 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.456 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-11-24T08:00:07.459 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.463 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.464 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.465 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.467 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.468 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.469 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-11-24T08:00:07.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.474 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-11-24T08:00:07.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-11-24T08:00:07.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-24T08:00:07.479 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-11-24T08:00:07.480 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.481 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.482 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.483 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.484 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.488 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.490 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.492 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.493 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.494 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.495 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.498 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.502 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.504 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.506 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.507 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.508 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.509 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.511 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.516 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.518 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.519 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.521 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-11-24T08:00:07.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-11-24T08:00:07.525 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.529 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.531 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.533 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.542 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.543 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.545 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.548 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.549 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.552 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.553 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.555 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:07.556 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.557 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.558 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-24T08:00:07.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.560 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.563 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.564 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.566 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.570 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.572 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.574 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.576 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.579 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.581 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.582 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.583 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.586 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.587 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.589 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-24T08:00:07.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.593 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.594 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.601 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.610 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.611 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.612 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-11-24T08:00:07.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.624 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.624 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.624 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.624 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.624 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-11-24T08:00:07.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.626 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.627 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.628 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.629 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.632 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.637 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.638 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.640 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.649 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.649 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.650 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.650 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.650 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.650 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.652 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-11-24T08:00:07.652 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.652 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.653 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.653 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-11-24T08:00:07.653 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-11-24T08:00:07.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-11-24T08:00:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.655 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-24T08:00:07.655 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.655 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.656 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.656 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.656 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.657 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.657 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.657 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.657 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.657 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.659 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.659 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.659 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.660 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.660 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.660 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.664 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.665 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-11-24T08:00:07.666 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.667 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.668 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.669 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.670 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.672 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.673 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.674 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.678 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.679 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.679 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.679 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.679 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.679 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.680 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.680 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-11-24T08:00:07.680 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-11-24T08:00:07.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.681 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.681 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-11-24T08:00:07.681 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.681 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.682 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.682 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.682 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.683 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.683 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-11-24T08:00:07.683 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-11-24T08:00:07.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.684 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.684 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.685 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.685 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.685 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.686 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.687 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.688 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.688 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-11-24T08:00:07.688 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-11-24T08:00:07.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.689 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.690 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.690 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.690 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.691 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.691 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.691 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.692 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-11-24T08:00:07.695 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-11-24T08:00:07.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.695 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.696 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.696 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.696 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.697 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.697 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.698 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.698 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-11-24T08:00:07.698 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-11-24T08:00:07.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.699 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.699 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.700 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.702 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-11-24T08:00:07.702 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-11-24T08:00:07.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.702 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.703 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.704 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.705 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-11-24T08:00:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.707 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.708 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.709 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.710 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.710 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-11-24T08:00:07.710 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-11-24T08:00:07.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.710 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.711 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.711 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.711 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-11-24T08:00:07.712 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.713 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-11-24T08:00:07.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-11-24T08:00:07.716 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.717 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.720 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.722 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.723 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.724 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.725 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.727 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-11-24T08:00:07.728 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.730 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.733 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.735 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.737 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-11-24T08:00:07.738 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.740 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.744 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.745 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.746 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.746 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.746 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.747 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-11-24T08:00:07.749 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.750 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:07.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.752 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.754 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.755 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.757 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.758 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.759 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.760 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-11-24T08:00:07.762 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.763 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.764 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.768 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-11-24T08:00:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-24T08:00:07.771 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.772 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.774 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.776 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.777 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-11-24T08:00:07.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-11-24T08:00:07.782 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.783 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.784 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.791 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.792 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.794 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.795 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-11-24T08:00:07.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.797 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.798 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.799 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.800 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.801 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.802 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.804 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.805 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.806 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.808 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.811 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.812 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.813 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.815 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.817 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-11-24T08:00:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.821 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.822 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-11-24T08:00:07.825 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.825 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.825 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.825 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-11-24T08:00:07.825 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.826 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.827 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.829 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.831 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.832 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.833 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.834 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.836 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.837 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.838 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-11-24T08:00:07.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.840 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.841 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.843 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: "modules": [ 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: "iostat", 2023-11-24T08:00:07.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nfs", 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: "restful" 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.846 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.847 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.848 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.849 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.850 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.852 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.853 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.854 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.855 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.856 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.859 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.861 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-11-24T08:00:07.862 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-11-24T08:00:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.865 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.866 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.867 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.868 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-11-24T08:00:07.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-11-24T08:00:07.870 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.871 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.874 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-11-24T08:00:07.875 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.875 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.875 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.875 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-11-24T08:00:07.875 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-11-24T08:00:07.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.876 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.877 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-11-24T08:00:07.878 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.879 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-11-24T08:00:07.880 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-11-24T08:00:07.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-24T08:00:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.884 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.885 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.887 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-11-24T08:00:07.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.889 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.890 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.891 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.892 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.893 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.895 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.896 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.898 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-11-24T08:00:07.902 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-11-24T08:00:07.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-24T08:00:07.904 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.905 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-11-24T08:00:07.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.912 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-11-24T08:00:07.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-24T08:00:07.914 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.915 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.917 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.919 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-11-24T08:00:07.920 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.921 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2023-11-24T08:00:07.923 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-11-24T08:00:07.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.925 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.926 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-24T08:00:07.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.928 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.930 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-11-24T08:00:07.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-11-24T08:00:07.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-24T08:00:07.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-11-24T08:00:07.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-11-24T08:00:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.942 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-11-24T08:00:07.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-11-24T08:00:07.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-11-24T08:00:07.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-11-24T08:00:07.949 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.950 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.953 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-24T08:00:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.955 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.956 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-11-24T08:00:07.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-11-24T08:00:07.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-24T08:00:07.965 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.966 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.970 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.971 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.972 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.973 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.974 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-24T08:00:07.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.976 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.977 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:07.978 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:07.979 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:07.980 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.981 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.989 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-11-24T08:00:07.991 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:07.992 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.993 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-11-24T08:00:07.995 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.996 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.997 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-24T08:00:07.998 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:07.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.003 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.004 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.005 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-11-24T08:00:08.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-11-24T08:00:08.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-24T08:00:08.009 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.016 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.019 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.020 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.021 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.024 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.026 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.027 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-11-24T08:00:08.029 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-11-24T08:00:08.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-11-24T08:00:08.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-24T08:00:08.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.034 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.039 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.040 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.041 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-24T08:00:08.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.044 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-24T08:00:08.047 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.048 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-24T08:00:08.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.054 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.056 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-11-24T08:00:08.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.059 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.060 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-11-24T08:00:08.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.063 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.066 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.067 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.071 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.072 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.073 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.074 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-24T08:00:08.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.076 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.077 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.078 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.083 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-24T08:00:08.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-24T08:00:08.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.087 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.090 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:08.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.093 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-24T08:00:08.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.097 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.098 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.100 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.102 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.103 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: "UNSAFE_TLS_v1_2": { 2023-11-24T08:00:08.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.105 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.106 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:08.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.111 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-11-24T08:00:08.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.115 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-11-24T08:00:08.116 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.118 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.120 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.121 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.122 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.125 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.126 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.131 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.132 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-11-24T08:00:08.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.134 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.136 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-11-24T08:00:08.140 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.141 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.142 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-11-24T08:00:08.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-11-24T08:00:08.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.145 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-11-24T08:00:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-11-24T08:00:08.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.150 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.152 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.153 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.154 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.157 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.159 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.162 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-11-24T08:00:08.163 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.165 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-24T08:00:08.166 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.167 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-11-24T08:00:08.169 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.171 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.172 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.173 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.177 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.178 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.179 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-11-24T08:00:08.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.183 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:08.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.185 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.186 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-11-24T08:00:08.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.190 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:08.192 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.193 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.194 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.195 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.199 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.201 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.202 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-11-24T08:00:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.206 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.207 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-11-24T08:00:08.208 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-11-24T08:00:08.211 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.213 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.214 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.215 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.216 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.218 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.220 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.224 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.226 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.230 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.231 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.232 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.233 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.234 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-11-24T08:00:08.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-11-24T08:00:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-24T08:00:08.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.238 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.239 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.242 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.244 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.245 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-11-24T08:00:08.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.250 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.251 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-11-24T08:00:08.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-11-24T08:00:08.253 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.254 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-24T08:00:08.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.256 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.258 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.259 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.262 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.265 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.266 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.268 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.272 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.275 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.276 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.277 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.278 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.279 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.280 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.282 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.283 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.286 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.288 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.294 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.297 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.298 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.300 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.302 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.303 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.304 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.306 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.307 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.310 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.311 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.312 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.313 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.316 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.317 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.320 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.322 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.326 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.328 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:08.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.331 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.334 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-24T08:00:08.335 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.338 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.339 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.340 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.341 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.343 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.344 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-11-24T08:00:08.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-11-24T08:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-11-24T08:00:08.350 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.351 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.352 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.354 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.355 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.356 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.359 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.360 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.361 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.363 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.364 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.365 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-11-24T08:00:08.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-11-24T08:00:08.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-11-24T08:00:08.377 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.379 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.381 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.382 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.383 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.385 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.388 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.389 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-11-24T08:00:08.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.394 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.394 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.396 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.398 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.400 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.402 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.404 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.407 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.408 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-11-24T08:00:08.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.410 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.411 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.413 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.417 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.418 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.420 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.421 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.423 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.424 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.425 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.430 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.432 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-11-24T08:00:08.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-24T08:00:08.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-11-24T08:00:08.435 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.437 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.439 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.440 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.441 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.442 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.443 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.446 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-11-24T08:00:08.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-11-24T08:00:08.450 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.451 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-11-24T08:00:08.452 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.454 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.455 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.457 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.458 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-11-24T08:00:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.460 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.461 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.462 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.463 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.465 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.466 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-11-24T08:00:08.467 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.468 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-11-24T08:00:08.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-11-24T08:00:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.472 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.473 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.475 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.476 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.477 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.479 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.482 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.484 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.485 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.486 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.487 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.489 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-11-24T08:00:08.490 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.491 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.492 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.495 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.496 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.497 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-11-24T08:00:08.500 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-11-24T08:00:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.504 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.506 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.507 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.510 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-11-24T08:00:08.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.515 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.516 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.522 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.523 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.524 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-11-24T08:00:08.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.526 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.527 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.529 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-11-24T08:00:08.531 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-11-24T08:00:08.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-11-24T08:00:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-11-24T08:00:08.552 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.554 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.555 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.558 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.560 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.562 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.563 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.564 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.565 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.567 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.568 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.569 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.570 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-11-24T08:00:08.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-24T08:00:08.572 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-11-24T08:00:08.575 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-11-24T08:00:08.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.578 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-11-24T08:00:08.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.580 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.581 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.582 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.583 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.584 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-11-24T08:00:08.585 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.586 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-11-24T08:00:08.588 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.589 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-11-24T08:00:08.590 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.591 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-11-24T08:00:08.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.594 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-11-24T08:00:08.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-11-24T08:00:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: "services": {}, 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: "always_on_modules": { 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: "octopus": [ 2023-11-24T08:00:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-11-24T08:00:08.598 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pacific": [ 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-11-24T08:00:08.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "quincy": [ 2023-11-24T08:00:08.600 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-11-24T08:00:08.601 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "reef": [ 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-11-24T08:00:08.602 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.603 INFO:teuthology.orchestra.run.smithi032.stdout: "last_failure_osd_epoch": 0, 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: "active_clients": [ 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.123:0", 2023-11-24T08:00:08.604 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2156780906 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "libcephsqlite", 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.123:0", 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3956559345 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-11-24T08:00:08.606 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.123:0", 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1328145929 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.607 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.123:0", 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3223130902 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:08.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-11-24T08:00:08.609 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-11-24T08:00:08.609 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f007406aed0 msgr2=0x7f007406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f007406aed0 0x7f007406d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f00881f7120 tx=0x7f007c009040 comp rx=0 tx=0).stop 2023-11-24T08:00:08.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 msgr2=0x7f0088152870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f0088152870 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f007800c550 tx=0x7f007800c910 comp rx=0 tx=0).stop 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 shutdown_connections 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f007406aed0 0x7f007406d390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0088152030 0x7f00880cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00880ae910 0x7f0088152870 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 --2- 172.21.15.32:0/2921524895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00880d0780 0x7f008815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 >> 172.21.15.32:0/2921524895 conn(0x7f00880730a0 msgr2=0x7f00880ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:08.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 shutdown_connections 2023-11-24T08:00:08.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:07.055+0000 7f008e698700 1 -- 172.21.15.32:0/2921524895 wait complete. 2023-11-24T08:00:08.616 DEBUG:tasks.check_counter:Getting stats from x 2023-11-24T08:00:08.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mgr.x.asok perf dump 2023-11-24T08:00:08.755 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-11-24T08:00:08.755 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-11-24T08:00:08.755 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 180, 2023-11-24T08:00:08.755 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 62, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2487680, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 91888, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 56, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 12, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.054337297, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.002128135, 2023-11-24T08:00:08.756 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.037361661, 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 49, 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.005482903, 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000111895 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.757 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2487680, 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 91888 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-11-24T08:00:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 402, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 214, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2768288, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 2555603, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 43, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 14, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.047672439, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.006677893, 2023-11-24T08:00:08.759 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.051372322, 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.002840470, 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 206, 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.009854759, 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000047838 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-11-24T08:00:08.760 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2768288, 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 2555472 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-11-24T08:00:08.761 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 566, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 342, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2504706, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 3189869, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 200, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 19, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.128924517, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.025598245, 2023-11-24T08:00:08.762 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.111805673, 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000018466, 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 331, 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.025724136, 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000077716 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-11-24T08:00:08.763 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2503824, 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 3189360 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-11-24T08:00:08.764 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client": { 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": { 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "lat": { 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "wrlat": { 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.766 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: "rdlat": { 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.767 INFO:teuthology.orchestra.run.smithi123.stdout: "fsync": { 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "mdavg": 0.000000000, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsqsum": 0, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "mdops": 0, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "readavg": 0.000000000, 2023-11-24T08:00:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: "readsqsum": 0, 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "rdops": 0, 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "writeavg": 0.000000000, 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "writesqsum": 0, 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "rdops": 0 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-Mgr": { 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.046439006, 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.023219503 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MgrStandby": { 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-balancer": { 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.771 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-cmd_finisher": { 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.772 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 17, 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000707413, 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000041612 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-crash": { 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.773 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-devicehealth": { 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.774 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-iostat": { 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.775 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-nfs": { 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.776 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-orchestrator": { 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.777 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-pg_autoscaler": { 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.778 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-progress": { 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.779 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-rbd_support": { 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.780 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-restful": { 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.781 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 17, 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003358935, 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000197584 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-status": { 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.782 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-telemetry": { 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.783 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-volumes": { 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-11-24T08:00:08.784 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000614997, 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000614997 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: "libcephsqlite_striper": { 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: "update_metadata": 8, 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: "update_allocated": 2, 2023-11-24T08:00:08.785 INFO:teuthology.orchestra.run.smithi123.stdout: "update_size": 6, 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "update_version": 8, 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "shrink": 0, 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "shrink_bytes": 0, 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "lock": 11, 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "unlock": 11 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "libcephsqlite_vfs": { 2023-11-24T08:00:08.786 INFO:teuthology.orchestra.run.smithi123.stdout: "op_open": { 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.476988826, 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.238494413 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "op_delete": { 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: "op_access": { 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: "op_fullpathname": { 2023-11-24T08:00:08.788 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: "op_currenttime": { 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4, 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_close": { 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_read": { 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20, 2023-11-24T08:00:08.790 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.006999836, 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000349991 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_write": { 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 85, 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.004999884, 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000058822 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.791 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_truncate": { 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_sync": { 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 24, 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.104997539, 2023-11-24T08:00:08.792 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004374897 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_filesize": { 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20, 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_lock": { 2023-11-24T08:00:08.793 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 27, 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.023999438, 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000888868 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_unlock": { 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 19, 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.020999509, 2023-11-24T08:00:08.794 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001105237 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_checkreservedlock": { 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9, 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_filecontrol": { 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 26, 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_sectorsize": { 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: "opf_devicecharacteristics": { 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 30, 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000999977, 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000033332 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-11-24T08:00:08.797 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-11-24T08:00:08.798 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-11-24T08:00:08.799 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-11-24T08:00:08.800 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-11-24T08:00:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 1616253, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 280, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 1496, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 17, 2023-11-24T08:00:08.802 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 30920, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 573, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-11-24T08:00:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 12896, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 310, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 0, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 0, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-11-24T08:00:08.804 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": { 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_hit": 0, 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_miss": 0 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: "objectcacher-libcephfs": { 2023-11-24T08:00:08.805 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_ops_hit": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_ops_miss": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_bytes_hit": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "cache_bytes_miss": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "data_read": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "data_written": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "data_flushed": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "data_overwritten_while_flushing": 0, 2023-11-24T08:00:08.806 INFO:teuthology.orchestra.run.smithi123.stdout: "write_ops_blocked": 0, 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "write_bytes_blocked": 0, 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "write_time_blocked": 0.000000000 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 0, 2023-11-24T08:00:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 0, 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 0, 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 0, 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.808 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 0, 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-11-24T08:00:08.809 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-11-24T08:00:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-11-24T08:00:08.811 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-11-24T08:00:08.812 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-11-24T08:00:08.813 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-11-24T08:00:08.814 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 8, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 35, 2023-11-24T08:00:08.815 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 8, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 8, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2023-11-24T08:00:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter-0x56167f9dee60": { 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 163, 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 2434738, 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 0, 2023-11-24T08:00:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 163, 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 163, 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.993976715, 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006098016 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-11-24T08:00:08.818 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 163, 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 307 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 163, 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 46, 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 117, 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-11-24T08:00:08.819 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 2, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 20, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 85, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-11-24T08:00:08.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 104, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 52, 2023-11-24T08:00:08.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 22, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 22, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T08:00:08.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-11-24T08:00:08.823 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-11-24T08:00:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 17, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 7, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 1, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 1, 2023-11-24T08:00:08.825 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter-0x56167f9df0e0": { 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-11-24T08:00:08.826 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 1, 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 39, 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 0, 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 1, 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2023-11-24T08:00:08.827 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000999977, 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000999977 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.828 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 1, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 1, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-11-24T08:00:08.829 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-11-24T08:00:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-11-24T08:00:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 1, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-11-24T08:00:08.832 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-11-24T08:00:08.833 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-11-24T08:00:08.834 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 0, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 0, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 1, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 1, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-11-24T08:00:08.835 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter-0x56167f9df900": { 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 0, 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 0, 2023-11-24T08:00:08.836 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 0, 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 0, 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-11-24T08:00:08.837 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 0, 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-11-24T08:00:08.838 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-11-24T08:00:08.839 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2023-11-24T08:00:08.840 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T08:00:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-11-24T08:00:08.842 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-11-24T08:00:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 0, 2023-11-24T08:00:08.844 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 0, 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter-0x56167f9dfb80": { 2023-11-24T08:00:08.845 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.846 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-11-24T08:00:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-11-24T08:00:08.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-11-24T08:00:08.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-11-24T08:00:08.853 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-11-24T08:00:08.853 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-11-24T08:00:08.853 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-11-24T08:00:08.853 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-11-24T08:00:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-11-24T08:00:08.855 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 0, 2023-11-24T08:00:08.856 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 0, 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_client_bytes": { 2023-11-24T08:00:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 134217728, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 52, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 241249, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 52, 2023-11-24T08:00:08.858 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 52, 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 241249, 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_client_messages": { 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 512, 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 52, 2023-11-24T08:00:08.860 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 52, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 52, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 52, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 52, 2023-11-24T08:00:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_mds_bytes": { 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.862 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 134217728, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 90, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 810228, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 90, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.863 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 90, 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 810228, 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_mds_messsages": { 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 128, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 90, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 90, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.865 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 90, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 90, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 90, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_mon_bytes": { 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 134217728, 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 66, 2023-11-24T08:00:08.867 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 464441, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 66, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 66, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 464441, 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_mon_messsages": { 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 128, 2023-11-24T08:00:08.869 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 66, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 66, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 66, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.870 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 66, 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 66, 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.871 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_osd_bytes": { 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 536870912, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 360, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 4244088, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 360, 2023-11-24T08:00:08.872 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 360, 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 4244088, 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-mgr_osd_messsages": { 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 8192, 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 360, 2023-11-24T08:00:08.874 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 360, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 360, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 360, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 360, 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mgr": { 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.876 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 357, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 669492, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 357, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 357, 2023-11-24T08:00:08.877 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 669492, 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.878 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mgr-0x56167babfd60": { 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 568, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 5760006, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 568, 2023-11-24T08:00:08.879 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 568, 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 5760006, 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-radosclient": { 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 175, 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 357801, 2023-11-24T08:00:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 175, 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 175, 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 357801, 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.882 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x56167f9de280": { 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.883 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 8, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 184480, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 8, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.884 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 8, 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 184480, 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.885 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x56167f9dfae0": { 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 9, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 314711, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 9, 2023-11-24T08:00:08.886 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 9, 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 314711, 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x56167fc48b40": { 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 10, 2023-11-24T08:00:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 314735, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 10, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 10, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 314735, 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.889 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.890 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-11-24T08:00:08.891 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes-0x56167f9de1e0": { 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1, 2023-11-24T08:00:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1, 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.894 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes-0x56167f9ded20": { 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 163, 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 2433049, 2023-11-24T08:00:08.895 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 163, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 163, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2433049, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.896 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes-0x56167f9dfa40": { 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-11-24T08:00:08.898 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes-0x56167fc48be0": { 2023-11-24T08:00:08.899 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.903 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops-0x56167f9dedc0": { 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 163, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 163, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 163, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 163, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 163, 2023-11-24T08:00:08.905 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops-0x56167f9df860": { 2023-11-24T08:00:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.907 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1, 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1, 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops-0x56167f9df9a0": { 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.909 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops-0x56167fc48c80": { 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-11-24T08:00:08.911 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-11-24T08:00:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-11-24T08:00:08.913 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-11-24T08:00:08.915 INFO:tasks.check_counter:Daemon mgr.x finisher-volumes.complete_latency.avgcount=1 2023-11-24T08:00:08.915 INFO:tasks.check_counter:Daemon mgr.x finisher-volumes.queue_len=0 2023-11-24T08:00:08.915 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mgr daemons: {'finisher-volumes.complete_latency.avgcount'} 2023-11-24T08:00:08.919 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-24T08:00:08.930 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-24T08:00:08.930 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T08:00:08.930 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:08.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17559 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T08:00:08.954 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T08:00:08.954 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:09.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17559 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T08:00:09.016 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T08:00:09.016 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:09.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18655 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T08:00:09.037 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T08:00:09.037 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:09.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18655 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T08:00:09.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.097 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T08:00:09.118 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:00:09.118 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:00:09.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:00:09.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:00:09.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-11-24T08:00:09.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.395 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T08:00:09.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-11-24T08:00:09.485 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:09.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:09.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.487 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T08:00:09.508 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:00:09.508 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:00:09.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:00:09.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:00:09.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-11-24T08:00:09.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.779 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T08:00:09.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-11-24T08:00:09.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:09.879 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:09.882 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-24T08:00:09.883 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T08:00:09.883 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:09.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17559 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-11-24T08:00:09.905 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-11-24T08:00:09.905 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:09.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17559 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-11-24T08:00:09.969 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T08:00:09.970 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:09.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18655 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-11-24T08:00:09.991 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-11-24T08:00:09.992 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:10.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18655 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-11-24T08:00:10.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.056 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T08:00:10.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-11-24T08:00:10.151 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:10.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:10.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.154 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-11-24T08:00:10.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-11-24T08:00:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:10.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:10.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.287 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-11-24T08:00:10.392 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:10.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:10.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.393 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-11-24T08:00:10.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:10.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-11-24T08:00:10.531 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:10.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:10.533 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T08:00:10.565 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T08:00:10.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:10.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.700+0000 7fceabe11700 1 Processor -- start 2023-11-24T08:00:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.701+0000 7fceabe11700 1 -- start start 2023-11-24T08:00:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.701+0000 7fceabe11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcea406eb70 0x7fcea4070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.701+0000 7fceabe11700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcea415b140 0x7fcea4071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.701+0000 7fceabe11700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40b3000 con 0x7fcea4158770 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.701+0000 7fceabe11700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40b32e0 con 0x7fcea406eb70 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.701+0000 7fceabe11700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40b35c0 con 0x7fcea415b140 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea93ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcea406eb70 0x7fcea4070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea9bad700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcea4158770 0x7fcea406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60762/0 (socket says 172.21.15.32:60762) 2023-11-24T08:00:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea9bad700 1 -- 172.21.15.32:0/2836665353 learned_addr learned my addr 172.21.15.32:0/2836665353 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fceaa3ae700 1 --2- 172.21.15.32:0/2836665353 >> v2:172.21.15.123:3300/0 conn(0x7fcea415b140 0x7fcea4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea93ac700 1 -- 172.21.15.32:0/2836665353 >> v2:172.21.15.123:3300/0 conn(0x7fcea415b140 msgr2=0x7fcea4071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea93ac700 1 --2- 172.21.15.32:0/2836665353 >> v2:172.21.15.123:3300/0 conn(0x7fcea415b140 0x7fcea4071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea93ac700 1 -- 172.21.15.32:0/2836665353 >> v1:172.21.15.32:6789/0 conn(0x7fcea4158770 legacy=0x7fcea406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea93ac700 1 -- 172.21.15.32:0/2836665353 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40b38a0 con 0x7fcea406eb70 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea93ac700 1 --2- 172.21.15.32:0/2836665353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4070bc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcea406d2f0 tx=0x7fce94009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203cd4d3727e225b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea8bab700 1 -- 172.21.15.32:0/2836665353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94011830 con 0x7fcea406eb70 2023-11-24T08:00:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea8bab700 1 -- 172.21.15.32:0/2836665353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94011990 con 0x7fcea406eb70 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.702+0000 7fcea8bab700 1 -- 172.21.15.32:0/2836665353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94011d20 con 0x7fcea406eb70 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 -- 172.21.15.32:0/2836665353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 msgr2=0x7fcea4070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 --2- 172.21.15.32:0/2836665353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4070bc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcea406d2f0 tx=0x7fce94009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 -- 172.21.15.32:0/2836665353 shutdown_connections 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 --2- 172.21.15.32:0/2836665353 >> v2:172.21.15.123:3300/0 conn(0x7fcea415b140 0x7fcea4071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 --2- 172.21.15.32:0/2836665353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4070bc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 -- 172.21.15.32:0/2836665353 >> 172.21.15.32:0/2836665353 conn(0x7fcea40b1740 msgr2=0x7fcea40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 -- 172.21.15.32:0/2836665353 shutdown_connections 2023-11-24T08:00:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.703+0000 7fceabe11700 1 -- 172.21.15.32:0/2836665353 wait complete. 2023-11-24T08:00:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.705+0000 7fceabe11700 1 Processor -- start 2023-11-24T08:00:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.705+0000 7fceabe11700 1 -- start start 2023-11-24T08:00:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4158770 0x7fcea40cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcea415b140 0x7fcea413bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ce2c0 con 0x7fcea406eb70 2023-11-24T08:00:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ce5a0 con 0x7fcea4158770 2023-11-24T08:00:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ce880 con 0x7fcea415b140 2023-11-24T08:00:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59056/0 (socket says 172.21.15.32:59056) 2023-11-24T08:00:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 -- 172.21.15.32:0/2031065983 learned_addr learned my addr 172.21.15.32:0/2031065983 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 -- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcea415b140 msgr2=0x7fcea413bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcea415b140 0x7fcea413bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 -- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4158770 msgr2=0x7fcea40cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea93ac700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4158770 0x7fcea40cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4158770 0x7fcea40cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 -- 172.21.15.32:0/2031065983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940094a0 con 0x7fcea406eb70 2023-11-24T08:00:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fcea9bad700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4154270 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c520 tx=0x7fcea000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0005d70 con 0x7fcea406eb70 2023-11-24T08:00:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea00070d0 con 0x7fcea406eb70 2023-11-24T08:00:10.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea40ceb60 con 0x7fcea406eb70 2023-11-24T08:00:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0007440 con 0x7fcea406eb70 2023-11-24T08:00:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.706+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea41547b0 con 0x7fcea406eb70 2023-11-24T08:00:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.707+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea4155fa0 con 0x7fcea406eb70 2023-11-24T08:00:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.708+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcea00075a0 con 0x7fcea406eb70 2023-11-24T08:00:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.708+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fce8806aed0 0x7fce8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.708+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcea0088670 con 0x7fcea406eb70 2023-11-24T08:00:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.708+0000 7fcea93ac700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fce8806aed0 0x7fce8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.709+0000 7fcea93ac700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fce8806aed0 0x7fce8806d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcea4151540 tx=0x7fce94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:10.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.713+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcea4155fa0 con 0x7fcea406eb70 2023-11-24T08:00:10.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.908+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcea40ce710 con 0x7fce8806aed0 2023-11-24T08:00:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.912+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2031065983 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50031 (secure 0 0 0) 0x7fcea40ce710 con 0x7fce8806aed0 2023-11-24T08:00:10.913 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fce8806aed0 msgr2=0x7fce8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fce8806aed0 0x7fce8806d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcea4151540 tx=0x7fce94008030 comp rx=0 tx=0).stop 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 msgr2=0x7fcea4154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4154270 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c520 tx=0x7fcea000c8e0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 shutdown_connections 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fce8806aed0 0x7fce8806d390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcea415b140 0x7fcea413bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcea406eb70 0x7fcea4154270 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 --2- 172.21.15.32:0/2031065983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcea4158770 0x7fcea40cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 >> 172.21.15.32:0/2031065983 conn(0x7fcea40b1740 msgr2=0x7fcea415c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 shutdown_connections 2023-11-24T08:00:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:10.913+0000 7fceabe11700 1 -- 172.21.15.32:0/2031065983 wait complete. 2023-11-24T08:00:10.918 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T08:00:10.927 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-11-24T08:00:10.552639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220520,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754200,"statfs":{"total":773094113280,"available":772868300800,"internally_reserved":0,"allocated":4431872,"data_stored":2432112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T07:59:54.780239+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:59:54.780239+0000","last_peered":"2023-11-24T07:59:54.780239+0000","last_clean":"2023-11-24T07:59:54.780239+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:59:54.780239+0000","last_undegraded":"2023-11-24T07:59:54.780239+0000","last_fullsized":"2023-11-24T07:59:54.780239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.39500000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.63}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.378},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.307},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.372}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.504}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.751},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.628},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.55000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.82299999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.88300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.254},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.439},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.48199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:10.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.053+0000 7fa9ad0c9700 1 Processor -- start 2023-11-24T08:00:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.053+0000 7fa9ad0c9700 1 -- start start 2023-11-24T08:00:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9ad0c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 0x7fa9a80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9ad0c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa9a815ea60 0x7fa9a815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9ad0c9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a815f520 con 0x7fa9a806c000 2023-11-24T08:00:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9ad0c9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a80b4230 con 0x7fa9a815ea60 2023-11-24T08:00:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9ad0c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80b3c40 con 0x7fa9a8144660 2023-11-24T08:00:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9a6d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 0x7fa9a80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9a659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa9a815ea60 0x7fa9a815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9a759e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa9a806c000 0x7fa9a806ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:47534/0 (socket says 172.21.15.32:47534) 2023-11-24T08:00:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9a759e700 1 -- 172.21.15.32:0/1509846760 learned_addr learned my addr 172.21.15.32:0/1509846760 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.054+0000 7fa9a6d9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 0x7fa9a80b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:54316/0 (socket says 172.21.15.32:54316) 2023-11-24T08:00:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a659c700 1 -- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 msgr2=0x7fa9a80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a659c700 1 --2- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 0x7fa9a80b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a659c700 1 -- 172.21.15.32:0/1509846760 >> v1:172.21.15.123:6789/0 conn(0x7fa9a806c000 legacy=0x7fa9a806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a659c700 1 -- 172.21.15.32:0/1509846760 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a815f800 con 0x7fa9a815ea60 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a6d9d700 1 --2- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 0x7fa9a80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a659c700 1 --2- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a815ea60 0x7fa9a815eee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa998009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8267bfc3492088ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a5d9b700 1 -- 172.21.15.32:0/1509846760 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011830 con 0x7fa9a815ea60 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a5d9b700 1 -- 172.21.15.32:0/1509846760 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998011990 con 0x7fa9a815ea60 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9a5d9b700 1 -- 172.21.15.32:0/1509846760 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998011d20 con 0x7fa9a815ea60 2023-11-24T08:00:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a815ea60 msgr2=0x7fa9a815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.055+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a815ea60 0x7fa9a815eee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa998009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.056+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/1509846760 shutdown_connections 2023-11-24T08:00:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.056+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a8144660 0x7fa9a80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.056+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/1509846760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a815ea60 0x7fa9a815eee0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.056+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/1509846760 >> 172.21.15.32:0/1509846760 conn(0x7fa9a8071ac0 msgr2=0x7fa9a8071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.056+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/1509846760 shutdown_connections 2023-11-24T08:00:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.056+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/1509846760 wait complete. 2023-11-24T08:00:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.057+0000 7fa9ad0c9700 1 Processor -- start 2023-11-24T08:00:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.057+0000 7fa9ad0c9700 1 -- start start 2023-11-24T08:00:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9ad0c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a806c000 0x7fa9a81f0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9ad0c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a8144660 0x7fa9a8140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9ad0c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 0x7fa9a8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9ad0c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a81ef560 con 0x7fa9a815ea60 2023-11-24T08:00:11.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9ad0c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a81402e0 con 0x7fa9a8144660 2023-11-24T08:00:11.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9ad0c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a81f0cc0 con 0x7fa9a806c000 2023-11-24T08:00:11.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 0x7fa9a8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 0x7fa9a8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59074/0 (socket says 172.21.15.32:59074) 2023-11-24T08:00:11.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 -- 172.21.15.32:0/2669898705 learned_addr learned my addr 172.21.15.32:0/2669898705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:11.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a6d9d700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a806c000 0x7fa9a81f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a659c700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a8144660 0x7fa9a8140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 -- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a806c000 msgr2=0x7fa9a81f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a806c000 0x7fa9a81f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 -- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a8144660 msgr2=0x7fa9a8140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a8144660 0x7fa9a8140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 -- 172.21.15.32:0/2669898705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0094a0 con 0x7fa9a815ea60 2023-11-24T08:00:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.058+0000 7fa9a759e700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 0x7fa9a8073f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000cbc0 tx=0x7fa9a000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.059+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a000a9e0 con 0x7fa9a815ea60 2023-11-24T08:00:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.059+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a000ab40 con 0x7fa9a815ea60 2023-11-24T08:00:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.059+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9980094a0 con 0x7fa9a815ea60 2023-11-24T08:00:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.059+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0012400 con 0x7fa9a815ea60 2023-11-24T08:00:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.059+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a8157a60 con 0x7fa9a815ea60 2023-11-24T08:00:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.060+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a81ef6d0 con 0x7fa9a815ea60 2023-11-24T08:00:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.060+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9a001b3b0 con 0x7fa9a815ea60 2023-11-24T08:00:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.061+0000 7fa98f7fe700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa99406aed0 0x7fa99406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.061+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9a0088df0 con 0x7fa9a815ea60 2023-11-24T08:00:11.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.061+0000 7fa9a6d9d700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa99406aed0 0x7fa99406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.061+0000 7fa9a6d9d700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa99406aed0 0x7fa99406d390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009d70 tx=0x7fa99c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.065+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9a0054240 con 0x7fa9a815ea60 2023-11-24T08:00:11.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.260+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9a80b4b90 con 0x7fa99406aed0 2023-11-24T08:00:11.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.264+0000 7fa98f7fe700 1 -- 172.21.15.32:0/2669898705 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50031 (secure 0 0 0) 0x7fa99c0099c0 con 0x7fa99406aed0 2023-11-24T08:00:11.265 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:11.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa99406aed0 msgr2=0x7fa99406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa99406aed0 0x7fa99406d390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009d70 tx=0x7fa99c009210 comp rx=0 tx=0).stop 2023-11-24T08:00:11.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 msgr2=0x7fa9a8073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 0x7fa9a8073f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000cbc0 tx=0x7fa9a000ced0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 shutdown_connections 2023-11-24T08:00:11.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa99406aed0 0x7fa99406d390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815ea60 0x7fa9a8073f90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9a806c000 0x7fa9a81f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 --2- 172.21.15.32:0/2669898705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a8144660 0x7fa9a8140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 >> 172.21.15.32:0/2669898705 conn(0x7fa9a8071ac0 msgr2=0x7fa9a8071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 shutdown_connections 2023-11-24T08:00:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.265+0000 7fa9ad0c9700 1 -- 172.21.15.32:0/2669898705 wait complete. 2023-11-24T08:00:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T08:00:11.279 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-11-24T08:00:10.552639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220520,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754200,"statfs":{"total":773094113280,"available":772868300800,"internally_reserved":0,"allocated":4431872,"data_stored":2432112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T07:59:54.780239+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:59:54.780239+0000","last_peered":"2023-11-24T07:59:54.780239+0000","last_clean":"2023-11-24T07:59:54.780239+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:59:54.780239+0000","last_undegraded":"2023-11-24T07:59:54.780239+0000","last_fullsized":"2023-11-24T07:59:54.780239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.39500000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.63}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.378},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.307},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.372}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.504}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.751},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.628},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.55000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.82299999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.88300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.254},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.439},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.48199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:11.280 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T08:00:11.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:11.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:11.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.405+0000 7f6f629a1700 1 Processor -- start 2023-11-24T08:00:11.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.405+0000 7f6f629a1700 1 -- start start 2023-11-24T08:00:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f629a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f5c1446a0 0x7f6f5c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f629a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c0af950 con 0x7f6f5c0adf00 2023-11-24T08:00:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f629a1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c0afc30 con 0x7f6f5c06c710 2023-11-24T08:00:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f629a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c144a80 con 0x7f6f5c1446a0 2023-11-24T08:00:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f5bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f5c1446a0 0x7f6f5c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f5b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f5c0adf00 0x7f6f5c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60790/0 (socket says 172.21.15.32:60790) 2023-11-24T08:00:11.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.406+0000 7f6f5b7fe700 1 -- 172.21.15.32:0/530177527 learned_addr learned my addr 172.21.15.32:0/530177527 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5affd700 1 -- 172.21.15.32:0/530177527 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407742112 0 0) 0x7f6f5c0af950 con 0x7f6f5c0adf00 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5affd700 1 -- 172.21.15.32:0/530177527 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f480035b0 con 0x7f6f5c0adf00 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5bfff700 1 -- 172.21.15.32:0/530177527 >> v1:172.21.15.123:6789/0 conn(0x7f6f5c06c710 legacy=0x7f6f5c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5bfff700 1 -- 172.21.15.32:0/530177527 >> v1:172.21.15.32:6789/0 conn(0x7f6f5c0adf00 legacy=0x7f6f5c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5bfff700 1 -- 172.21.15.32:0/530177527 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0aff10 con 0x7f6f5c1446a0 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5bfff700 1 --2- 172.21.15.32:0/530177527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c0b5ae0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009840 tx=0x7f6f4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9655034ef728a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5affd700 1 -- 172.21.15.32:0/530177527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c011830 con 0x7f6f5c1446a0 2023-11-24T08:00:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5affd700 1 -- 172.21.15.32:0/530177527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c011990 con 0x7f6f5c1446a0 2023-11-24T08:00:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f5affd700 1 -- 172.21.15.32:0/530177527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c011d00 con 0x7f6f5c1446a0 2023-11-24T08:00:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.407+0000 7f6f629a1700 1 -- 172.21.15.32:0/530177527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 msgr2=0x7f6f5c0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.408+0000 7f6f629a1700 1 --2- 172.21.15.32:0/530177527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c0b5ae0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009840 tx=0x7f6f4c009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.408+0000 7f6f629a1700 1 -- 172.21.15.32:0/530177527 shutdown_connections 2023-11-24T08:00:11.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.408+0000 7f6f629a1700 1 --2- 172.21.15.32:0/530177527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c0b5ae0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.408+0000 7f6f629a1700 1 -- 172.21.15.32:0/530177527 >> 172.21.15.32:0/530177527 conn(0x7f6f5c071b90 msgr2=0x7f6f5c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.408+0000 7f6f629a1700 1 -- 172.21.15.32:0/530177527 shutdown_connections 2023-11-24T08:00:11.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.408+0000 7f6f629a1700 1 -- 172.21.15.32:0/530177527 wait complete. 2023-11-24T08:00:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.409+0000 7f6f629a1700 1 Processor -- start 2023-11-24T08:00:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.409+0000 7f6f629a1700 1 -- start start 2023-11-24T08:00:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f629a1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f5c06c710 0x7f6f5c13a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f629a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f5c0adf00 0x7f6f5c1f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f629a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c1e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f629a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c13a8d0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f629a1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1e6ff0 con 0x7f6f5c0adf00 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f629a1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1e63e0 con 0x7f6f5c06c710 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c1e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c1e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59088/0 (socket says 172.21.15.32:59088) 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 -- 172.21.15.32:0/2907253928 learned_addr learned my addr 172.21.15.32:0/2907253928 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f5bfff700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f5c06c710 0x7f6f5c13a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f5b7fe700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f5c0adf00 0x7f6f5c1f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 -- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f5c06c710 msgr2=0x7f6f5c13a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f5c06c710 0x7f6f5c13a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 -- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f5c0adf00 msgr2=0x7f6f5c1f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f5c0adf00 0x7f6f5c1f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 -- 172.21.15.32:0/2907253928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0094a0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f5bfff700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f5c06c710 0x7f6f5c13a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:11.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f60f3e700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c1e7480 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400cbc0 tx=0x7f6f5400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.410+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54003c00 con 0x7f6f5c1446a0 2023-11-24T08:00:11.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.411+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f54003d60 con 0x7f6f5c1446a0 2023-11-24T08:00:11.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.411+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c1e66c0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.411+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400abf0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.411+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c1e6c80 con 0x7f6f5c1446a0 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.412+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c0753f0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.412+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f5400ad50 con 0x7f6f5c1446a0 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.412+0000 7f6f58ff9700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6f3c06aed0 0x7f6f3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.413+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6f540893f0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.413+0000 7f6f5bfff700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6f3c06aed0 0x7f6f3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.413+0000 7f6f5bfff700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6f3c06aed0 0x7f6f3c06d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009fd0 tx=0x7f6f4c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.417+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f540547a0 con 0x7f6f5c1446a0 2023-11-24T08:00:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.613+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f5c0b6020 con 0x7f6f3c06aed0 2023-11-24T08:00:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.616+0000 7f6f58ff9700 1 -- 172.21.15.32:0/2907253928 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50031 (secure 0 0 0) 0x7f6f4c0117b0 con 0x7f6f3c06aed0 2023-11-24T08:00:11.617 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6f3c06aed0 msgr2=0x7f6f3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6f3c06aed0 0x7f6f3c06d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009fd0 tx=0x7f6f4c01c040 comp rx=0 tx=0).stop 2023-11-24T08:00:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 msgr2=0x7f6f5c1e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c1e7480 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400cbc0 tx=0x7f6f5400cf80 comp rx=0 tx=0).stop 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 shutdown_connections 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f6f3c06aed0 0x7f6f3c06d390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f5c1446a0 0x7f6f5c1e7480 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6f5c06c710 0x7f6f5c13a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 --2- 172.21.15.32:0/2907253928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f5c0adf00 0x7f6f5c1f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.617+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 >> 172.21.15.32:0/2907253928 conn(0x7f6f5c071b90 msgr2=0x7f6f5c0b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.618+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 shutdown_connections 2023-11-24T08:00:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.618+0000 7f6f629a1700 1 -- 172.21.15.32:0/2907253928 wait complete. 2023-11-24T08:00:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T08:00:11.631 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-11-24T08:00:10.552639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220520,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754200,"statfs":{"total":773094113280,"available":772868300800,"internally_reserved":0,"allocated":4431872,"data_stored":2432112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T07:59:54.780239+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:59:54.780239+0000","last_peered":"2023-11-24T07:59:54.780239+0000","last_clean":"2023-11-24T07:59:54.780239+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:59:54.780239+0000","last_undegraded":"2023-11-24T07:59:54.780239+0000","last_fullsized":"2023-11-24T07:59:54.780239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.499},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.39500000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.63}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.378},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.307},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.372}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.504}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.751},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.628},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.55000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.82299999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.88300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.254},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.439},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.48199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:11.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.794+0000 7f320f485700 1 Processor -- start 2023-11-24T08:00:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320f485700 1 -- start start 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320f485700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f320813f170 0x7f320806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320f485700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3208160610 0x7f32080cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320f485700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3208160a10 con 0x7f320806c540 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320f485700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320806c9d0 con 0x7f3208160610 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320f485700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320815c5b0 con 0x7f320813f170 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.795+0000 7f320da22700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3208160610 0x7f32080cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3208160610 0x7f32080cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59096/0 (socket says 172.21.15.32:59096) 2023-11-24T08:00:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 -- 172.21.15.32:0/2907567817 learned_addr learned my addr 172.21.15.32:0/2907567817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320d221700 1 --2- 172.21.15.32:0/2907567817 >> v2:172.21.15.123:3300/0 conn(0x7f320813f170 0x7f320806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 -- 172.21.15.32:0/2907567817 >> v2:172.21.15.123:3300/0 conn(0x7f320813f170 msgr2=0x7f320806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 --2- 172.21.15.32:0/2907567817 >> v2:172.21.15.123:3300/0 conn(0x7f320813f170 0x7f320806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 -- 172.21.15.32:0/2907567817 >> v1:172.21.15.123:6789/0 conn(0x7f320806c540 legacy=0x7f320815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 -- 172.21.15.32:0/2907567817 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320815c880 con 0x7f3208160610 2023-11-24T08:00:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f320da22700 1 --2- 172.21.15.32:0/2907567817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f32080cf5a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3208072f60 tx=0x7f320000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc6f713d7f554d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f31fbfff700 1 -- 172.21.15.32:0/2907567817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000e070 con 0x7f3208160610 2023-11-24T08:00:11.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f31fbfff700 1 -- 172.21.15.32:0/2907567817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320000b9c0 con 0x7f3208160610 2023-11-24T08:00:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.796+0000 7f31fbfff700 1 -- 172.21.15.32:0/2907567817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000bd70 con 0x7f3208160610 2023-11-24T08:00:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 -- 172.21.15.32:0/2907567817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 msgr2=0x7f32080cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 --2- 172.21.15.32:0/2907567817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f32080cf5a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3208072f60 tx=0x7f320000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 -- 172.21.15.32:0/2907567817 shutdown_connections 2023-11-24T08:00:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 --2- 172.21.15.32:0/2907567817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f32080cf5a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 --2- 172.21.15.32:0/2907567817 >> v2:172.21.15.123:3300/0 conn(0x7f320813f170 0x7f320806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 -- 172.21.15.32:0/2907567817 >> 172.21.15.32:0/2907567817 conn(0x7f3208071ac0 msgr2=0x7f3208071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 -- 172.21.15.32:0/2907567817 shutdown_connections 2023-11-24T08:00:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.797+0000 7f320f485700 1 -- 172.21.15.32:0/2907567817 wait complete. 2023-11-24T08:00:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.798+0000 7f320f485700 1 Processor -- start 2023-11-24T08:00:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.798+0000 7f320f485700 1 -- start start 2023-11-24T08:00:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f320806c540 0x7f320806e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f320813f170 0x7f32081f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f320813e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320813e8c0 con 0x7f3208160610 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080b53a0 con 0x7f320813f170 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32080b5680 con 0x7f320806c540 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f320813e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f320813e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59102/0 (socket says 172.21.15.32:59102) 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 -- 172.21.15.32:0/1935695475 learned_addr learned my addr 172.21.15.32:0/1935695475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320d221700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f320806c540 0x7f320806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320ca20700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f320813f170 0x7f32081f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 -- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f320806c540 msgr2=0x7f320806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f320806c540 0x7f320806e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 -- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f320813f170 msgr2=0x7f32081f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f320813f170 0x7f32081f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 -- 172.21.15.32:0/1935695475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200009b30 con 0x7f3208160610 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320da22700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f320813e380 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3208160ba0 tx=0x7f3200004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200004880 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32000099b0 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32080b5960 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.799+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320001b670 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.800+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32080ae870 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.801+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3208067f60 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.801+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f320001b7d0 con 0x7f3208160610 2023-11-24T08:00:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.801+0000 7f31f9ffb700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f31ec06aed0 0x7f31ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.802+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f320008ab00 con 0x7f3208160610 2023-11-24T08:00:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.802+0000 7f320d221700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f31ec06aed0 0x7f31ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.802+0000 7f320d221700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f31ec06aed0 0x7f31ec06d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f320815ba30 tx=0x7f31fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:11.806+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3200055ea0 con 0x7f3208160610 2023-11-24T08:00:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.008+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f320806c200 con 0x7f3208160610 2023-11-24T08:00:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.009+0000 7f31f9ffb700 1 -- 172.21.15.32:0/1935695475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+12972 (secure 0 0 0) 0x7f32000594c0 con 0x7f3208160610 2023-11-24T08:00:12.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:12.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"0b3006fa-b594-4439-bcd2-bb19d4a76534","created":"2023-11-24T07:58:20.838548+0000","modified":"2023-11-24T07:59:55.775681+0000","last_up_change":"2023-11-24T07:58:30.983512+0000","last_in_change":"2023-11-24T07:58:23.336987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:58:32.587117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de2c961b-886f-4896-b585-53b827ca14fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6817","nonce":843272889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6819","nonce":843272889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6823","nonce":843272889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":843272889},{"type":"v1","addr":"172.21.15.32:6821","nonce":843272889}]},"public_addr":"172.21.15.32:6817/843272889","cluster_addr":"172.21.15.32:6819/843272889","heartbeat_back_addr":"172.21.15.32:6823/843272889","heartbeat_front_addr":"172.21.15.32:6821/843272889","state":["exists","up"]},{"osd":1,"uuid":"c549a8a2-fb37-4671-874e-5f3bfe42a83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6801","nonce":835572876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6803","nonce":835572876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6807","nonce":835572876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":835572876},{"type":"v1","addr":"172.21.15.32:6805","nonce":835572876}]},"public_addr":"172.21.15.32:6801/835572876","cluster_addr":"172.21.15.32:6803/835572876","heartbeat_back_addr":"172.21.15.32:6807/835572876","heartbeat_front_addr":"172.21.15.32:6805/835572876","state":["exists","up"]},{"osd":2,"uuid":"a71282b6-7723-43e7-aab3-5a78e9fd15ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6825","nonce":3901267389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6827","nonce":3901267389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6831","nonce":3901267389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3901267389},{"type":"v1","addr":"172.21.15.32:6829","nonce":3901267389}]},"public_addr":"172.21.15.32:6825/3901267389","cluster_addr":"172.21.15.32:6827/3901267389","heartbeat_back_addr":"172.21.15.32:6831/3901267389","heartbeat_front_addr":"172.21.15.32:6829/3901267389","state":["exists","up"]},{"osd":3,"uuid":"e6eee010-f6ae-485d-b7d7-7ea8d03473d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6809","nonce":2834432589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6811","nonce":2834432589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6815","nonce":2834432589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2834432589},{"type":"v1","addr":"172.21.15.32:6813","nonce":2834432589}]},"public_addr":"172.21.15.32:6809/2834432589","cluster_addr":"172.21.15.32:6811/2834432589","heartbeat_back_addr":"172.21.15.32:6815/2834432589","heartbeat_front_addr":"172.21.15.32:6813/2834432589","state":["exists","up"]},{"osd":4,"uuid":"eba71c79-ee64-4d18-9f0e-ead114080731","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6809","nonce":1395851986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6811","nonce":1395851986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6815","nonce":1395851986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1395851986},{"type":"v1","addr":"172.21.15.123:6813","nonce":1395851986}]},"public_addr":"172.21.15.123:6809/1395851986","cluster_addr":"172.21.15.123:6811/1395851986","heartbeat_back_addr":"172.21.15.123:6815/1395851986","heartbeat_front_addr":"172.21.15.123:6813/1395851986","state":["exists","up"]},{"osd":5,"uuid":"746138ca-9c6f-4895-a296-bd6778103fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6822","nonce":3666097374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6826","nonce":3666097374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6831","nonce":3666097374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3666097374},{"type":"v1","addr":"172.21.15.123:6829","nonce":3666097374}]},"public_addr":"172.21.15.123:6822/3666097374","cluster_addr":"172.21.15.123:6826/3666097374","heartbeat_back_addr":"172.21.15.123:6831/3666097374","heartbeat_front_addr":"172.21.15.123:6829/3666097374","state":["exists","up"]},{"osd":6,"uuid":"86f44387-1510-4912-b4bc-3ee6550bf505","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6801","nonce":2373254053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6803","nonce":2373254053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6807","nonce":2373254053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2373254053},{"type":"v1","addr":"172.21.15.123:6805","nonce":2373254053}]},"public_addr":"172.21.15.123:6801/2373254053","cluster_addr":"172.21.15.123:6803/2373254053","heartbeat_back_addr":"172.21.15.123:6807/2373254053","heartbeat_front_addr":"172.21.15.123:6805/2373254053","state":["exists","up"]},{"osd":7,"uuid":"c822d5cc-2051-4923-8ddb-32ead3b8cf10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6817","nonce":3074243597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6819","nonce":3074243597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6827","nonce":3074243597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":3074243597},{"type":"v1","addr":"172.21.15.123:6823","nonce":3074243597}]},"public_addr":"172.21.15.123:6817/3074243597","cluster_addr":"172.21.15.123:6819/3074243597","heartbeat_back_addr":"172.21.15.123:6827/3074243597","heartbeat_front_addr":"172.21.15.123:6823/3074243597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538029+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.459381+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:29.945108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.476893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535736+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.535640+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.538163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:58:28.505693+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/598023464":"2023-11-25T07:59:52.705443+0000","172.21.15.32:6833/598023464":"2023-11-25T07:59:52.705443+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f31ec06aed0 msgr2=0x7f31ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f31ec06aed0 0x7f31ec06d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f320815ba30 tx=0x7f31fc008040 comp rx=0 tx=0).stop 2023-11-24T08:00:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 msgr2=0x7f320813e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f320813e380 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3208160ba0 tx=0x7f3200004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 shutdown_connections 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f31ec06aed0 0x7f31ec06d390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3208160610 0x7f320813e380 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f320806c540 0x7f320806e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 --2- 172.21.15.32:0/1935695475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f320813f170 0x7f32081f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 >> 172.21.15.32:0/1935695475 conn(0x7f3208071ac0 msgr2=0x7f32080b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 shutdown_connections 2023-11-24T08:00:12.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:12.011+0000 7f320f485700 1 -- 172.21.15.32:0/1935695475 wait complete. 2023-11-24T08:00:13.027 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T08:00:13.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 Processor -- start 2023-11-24T08:00:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 -- start start 2023-11-24T08:00:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7aa0143b80 0x7f7aa006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7aa00ce090 0x7f7aa006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa006ee30 con 0x7f7aa0144190 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa0143f60 con 0x7f7aa00ce090 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7aa4de2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa00b3f30 con 0x7f7aa0143b80 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7aa00ce090 0x7f7aa006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7aa0144190 0x7f7aa015e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60820/0 (socket says 172.21.15.32:60820) 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9ed9d700 1 -- 172.21.15.32:0/1082605527 learned_addr learned my addr 172.21.15.32:0/1082605527 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9e59c700 1 --2- 172.21.15.32:0/1082605527 >> v2:172.21.15.123:3300/0 conn(0x7f7aa0143b80 0x7f7aa006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1082605527 >> v2:172.21.15.123:3300/0 conn(0x7f7aa0143b80 msgr2=0x7f7aa006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9dd9b700 1 --2- 172.21.15.32:0/1082605527 >> v2:172.21.15.123:3300/0 conn(0x7f7aa0143b80 0x7f7aa006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1082605527 >> v1:172.21.15.32:6789/0 conn(0x7f7aa0144190 legacy=0x7f7aa015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1082605527 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa00b4210 con 0x7f7aa00ce090 2023-11-24T08:00:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9d59a700 1 -- 172.21.15.32:0/1082605527 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166604551 0 0) 0x7f7aa006ee30 con 0x7f7aa0144190 2023-11-24T08:00:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.169+0000 7f7a9dd9b700 1 --2- 172.21.15.32:0/1082605527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa00ce090 0x7f7aa006c540 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0062c70 tx=0x7f7a94009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ca45ac222025e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7a9d59a700 1 -- 172.21.15.32:0/1082605527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011830 con 0x7f7aa00ce090 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7a9d59a700 1 -- 172.21.15.32:0/1082605527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a94011990 con 0x7f7aa00ce090 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7a9d59a700 1 -- 172.21.15.32:0/1082605527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011d20 con 0x7f7aa00ce090 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1082605527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa00ce090 msgr2=0x7f7aa006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 --2- 172.21.15.32:0/1082605527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa00ce090 0x7f7aa006c540 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0062c70 tx=0x7f7a94009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1082605527 shutdown_connections 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 --2- 172.21.15.32:0/1082605527 >> v2:172.21.15.123:3300/0 conn(0x7f7aa0143b80 0x7f7aa006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 --2- 172.21.15.32:0/1082605527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa00ce090 0x7f7aa006c540 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1082605527 >> 172.21.15.32:0/1082605527 conn(0x7f7aa0071b50 msgr2=0x7f7aa0071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1082605527 shutdown_connections 2023-11-24T08:00:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.170+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1082605527 wait complete. 2023-11-24T08:00:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 Processor -- start 2023-11-24T08:00:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 -- start start 2023-11-24T08:00:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7aa00ce090 0x7f7aa0155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa0143b80 0x7f7aa01f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7aa0144190 0x7f7aa0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01f50c0 con 0x7f7aa0143b80 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01f4c30 con 0x7f7aa00ce090 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.172+0000 7f7aa4de2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01f4020 con 0x7f7aa0144190 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa0143b80 0x7f7aa01f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa0143b80 0x7f7aa01f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59112/0 (socket says 172.21.15.32:59112) 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1133330427 learned_addr learned my addr 172.21.15.32:0/1133330427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9e59c700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7aa00ce090 0x7f7aa0155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9ed9d700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7aa0144190 0x7f7aa0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7aa0144190 msgr2=0x7f7aa0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7aa0144190 0x7f7aa0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7aa00ce090 msgr2=0x7f7aa0155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7aa00ce090 0x7f7aa0155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 -- 172.21.15.32:0/1133330427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940094a0 con 0x7f7aa0143b80 2023-11-24T08:00:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9e59c700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7aa00ce090 0x7f7aa0155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a9dd9b700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa0143b80 0x7f7aa01f37d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009b60 tx=0x7f7a94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940036a0 con 0x7f7aa0143b80 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9401f070 con 0x7f7aa0143b80 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94003a60 con 0x7f7aa0143b80 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa01f4300 con 0x7f7aa0143b80 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.173+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa01f48c0 con 0x7f7aa0143b80 2023-11-24T08:00:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.174+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7aa00b4ab0 con 0x7f7aa0143b80 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a94003bc0 con 0x7f7aa0143b80 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a8effd700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a8006aed0 0x7f7a8006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a94021070 con 0x7f7aa0143b80 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a8effd700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f7a80070d30 0x7f7a80073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 --> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a80073820 con 0x7f7a80070d30 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a9408f2c0 con 0x7f7aa0143b80 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a9e59c700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a8006aed0 0x7f7a8006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.175+0000 7f7a9ed9d700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f7a80070d30 0x7f7a80073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.176+0000 7f7a9ed9d700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f7a80070d30 0x7f7a80073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.176+0000 7f7a9e59c700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a8006aed0 0x7f7a8006d390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7a900099f0 tx=0x7f7a90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.176+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== osd.0 v2:172.21.15.32:6816/843272889 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7a80073820 con 0x7f7a80070d30 2023-11-24T08:00:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 --> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7aa00b47d0 con 0x7f7a80070d30 2023-11-24T08:00:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7a8effd700 1 -- 172.21.15.32:0/1133330427 <== osd.0 v2:172.21.15.32:6816/843272889 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7a980070b0 con 0x7f7a80070d30 2023-11-24T08:00:13.195 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:13.195 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:13.195 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:13.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f7a80070d30 msgr2=0x7f7a80073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:6816/843272889,v1:172.21.15.32:6817/843272889] conn(0x7f7a80070d30 0x7f7a80073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a8006aed0 msgr2=0x7f7a8006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7a8006aed0 0x7f7a8006d390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7a900099f0 tx=0x7f7a90008040 comp rx=0 tx=0).stop 2023-11-24T08:00:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa0143b80 msgr2=0x7f7aa01f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 --2- 172.21.15.32:0/1133330427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7aa0143b80 0x7f7aa01f37d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009b60 tx=0x7f7a94004750 comp rx=0 tx=0).stop 2023-11-24T08:00:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7a9ed9d700 1 -- 172.21.15.32:0/1133330427 reap_dead start 2023-11-24T08:00:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 shutdown_connections 2023-11-24T08:00:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 >> 172.21.15.32:0/1133330427 conn(0x7f7aa0071b50 msgr2=0x7f7aa00ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 shutdown_connections 2023-11-24T08:00:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.192+0000 7f7aa4de2700 1 -- 172.21.15.32:0/1133330427 wait complete. 2023-11-24T08:00:13.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T08:00:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.330+0000 7fb04c67e700 1 Processor -- start 2023-11-24T08:00:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.330+0000 7fb04c67e700 1 -- start start 2023-11-24T08:00:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb04415e080 0x7fb0440cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0440ce4d0 0x7fb0440d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb0440ce910 0x7fb0440d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04415e460 con 0x7fb0440ce4d0 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0440d0150 con 0x7fb0440ce910 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440cee50 con 0x7fb04415e080 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0440ce4d0 0x7fb0440d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0440ce4d0 0x7fb0440d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59120/0 (socket says 172.21.15.32:59120) 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 -- 172.21.15.32:0/3759421247 learned_addr learned my addr 172.21.15.32:0/3759421247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04ac1b700 1 --2- 172.21.15.32:0/3759421247 >> v2:172.21.15.123:3300/0 conn(0x7fb0440ce910 0x7fb0440d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04a41a700 1 --2- 172.21.15.32:0/3759421247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb04415e080 0x7fb0440cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 -- 172.21.15.32:0/3759421247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb04415e080 msgr2=0x7fb0440cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 --2- 172.21.15.32:0/3759421247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb04415e080 0x7fb0440cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 -- 172.21.15.32:0/3759421247 >> v2:172.21.15.123:3300/0 conn(0x7fb0440ce910 msgr2=0x7fb0440d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 --2- 172.21.15.32:0/3759421247 >> v2:172.21.15.123:3300/0 conn(0x7fb0440ce910 0x7fb0440d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 -- 172.21.15.32:0/3759421247 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440cf130 con 0x7fb0440ce4d0 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049c19700 1 --2- 172.21.15.32:0/3759421247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0440ce4d0 0x7fb0440d1be0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb04406d190 tx=0x7fb040009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fa50263f5fd5e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049418700 1 -- 172.21.15.32:0/3759421247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000b870 con 0x7fb0440ce4d0 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049418700 1 -- 172.21.15.32:0/3759421247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04000b9d0 con 0x7fb0440ce4d0 2023-11-24T08:00:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb049418700 1 -- 172.21.15.32:0/3759421247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000bd60 con 0x7fb0440ce4d0 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- 172.21.15.32:0/3759421247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0440ce4d0 msgr2=0x7fb0440d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- 172.21.15.32:0/3759421247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0440ce4d0 0x7fb0440d1be0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb04406d190 tx=0x7fb040009bd0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- 172.21.15.32:0/3759421247 shutdown_connections 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- 172.21.15.32:0/3759421247 >> v2:172.21.15.123:3300/0 conn(0x7fb0440ce910 0x7fb0440d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- 172.21.15.32:0/3759421247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb04415e080 0x7fb0440cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 --2- 172.21.15.32:0/3759421247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0440ce4d0 0x7fb0440d1be0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- 172.21.15.32:0/3759421247 >> 172.21.15.32:0/3759421247 conn(0x7fb044073130 msgr2=0x7fb04409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.332+0000 7fb04c67e700 1 -- 172.21.15.32:0/3759421247 shutdown_connections 2023-11-24T08:00:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.333+0000 7fb04c67e700 1 -- 172.21.15.32:0/3759421247 wait complete. 2023-11-24T08:00:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.334+0000 7fb04c67e700 1 Processor -- start 2023-11-24T08:00:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.334+0000 7fb04c67e700 1 -- start start 2023-11-24T08:00:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0440ce4d0 0x7fb0440b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440ce910 0x7fb0440b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 0x7fb0440c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b6680 con 0x7fb04415e080 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440c8d20 con 0x7fb0440ce910 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440c9000 con 0x7fb0440ce4d0 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 0x7fb0440c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 0x7fb0440c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59130/0 (socket says 172.21.15.32:59130) 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 -- 172.21.15.32:0/584563862 learned_addr learned my addr 172.21.15.32:0/584563862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04a41a700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0440ce4d0 0x7fb0440b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb049c19700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440ce910 0x7fb0440b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 -- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0440ce4d0 msgr2=0x7fb0440b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0440ce4d0 0x7fb0440b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 -- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440ce910 msgr2=0x7fb0440b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440ce910 0x7fb0440b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 -- 172.21.15.32:0/584563862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009550 con 0x7fb04415e080 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04ac1b700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 0x7fb0440c74c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007e30 tx=0x7fb03c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00ed30 con 0x7fb04415e080 2023-11-24T08:00:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00ee90 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440c92e0 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c014660 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.335+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0440c9ae0 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.336+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0440cf8c0 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.337+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb03c0147c0 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.337+0000 7fb03affd700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb02806af00 0x7fb02806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.337+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb03c089850 con 0x7fb04415e080 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.337+0000 7fb04a41a700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb02806af00 0x7fb02806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.338+0000 7fb04a41a700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb02806af00 0x7fb02806d3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb034006b20 tx=0x7fb034008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.343+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb03c054c90 con 0x7fb04415e080 2023-11-24T08:00:13.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.541+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0440c8e90 con 0x7fb02806af00 2023-11-24T08:00:13.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.542+0000 7fb03affd700 1 -- 172.21.15.32:0/584563862 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb0440c8e90 con 0x7fb02806af00 2023-11-24T08:00:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb02806af00 msgr2=0x7fb02806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb02806af00 0x7fb02806d3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb034006b20 tx=0x7fb034008040 comp rx=0 tx=0).stop 2023-11-24T08:00:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 msgr2=0x7fb0440c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 0x7fb0440c74c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007e30 tx=0x7fb03c00e3c0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 shutdown_connections 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb02806af00 0x7fb02806d3c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb04415e080 0x7fb0440c74c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb0440ce4d0 0x7fb0440b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 --2- 172.21.15.32:0/584563862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0440ce910 0x7fb0440b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 >> 172.21.15.32:0/584563862 conn(0x7fb044073130 msgr2=0x7fb04415d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 shutdown_connections 2023-11-24T08:00:13.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.545+0000 7fb04c67e700 1 -- 172.21.15.32:0/584563862 wait complete. 2023-11-24T08:00:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T08:00:13.559 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T08:00:13.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.687+0000 7f7e0e820700 1 Processor -- start 2023-11-24T08:00:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.687+0000 7f7e0e820700 1 -- start start 2023-11-24T08:00:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e0e820700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e0806c710 0x7f7e0806dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e0e820700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 0x7f7e0806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e0e820700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e081624c0 con 0x7f7e0815e0b0 2023-11-24T08:00:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e0e820700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0815e490 con 0x7f7e0806c710 2023-11-24T08:00:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e0e820700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08161b90 con 0x7f7e080b5b60 2023-11-24T08:00:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e07fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e0815e0b0 0x7f7e0806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60846/0 (socket says 172.21.15.32:60846) 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e07fff700 1 -- 172.21.15.32:0/3689287330 learned_addr learned my addr 172.21.15.32:0/3689287330 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868876257 0 0) 0x7f7e081624c0 con 0x7f7e0815e0b0 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df40035b0 con 0x7f7e0815e0b0 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 0x7f7e0806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e077fe700 1 --2- 172.21.15.32:0/3689287330 >> v2:172.21.15.123:3300/0 conn(0x7f7e0806c710 0x7f7e0806dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245611058 0 0) 0x7f7df40035b0 con 0x7f7e0815e0b0 2023-11-24T08:00:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 msgr2=0x7f7e0806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 0x7f7e0806e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 >> v2:172.21.15.123:3300/0 conn(0x7f7e0806c710 msgr2=0x7f7e0806dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 --2- 172.21.15.32:0/3689287330 >> v2:172.21.15.123:3300/0 conn(0x7f7e0806c710 0x7f7e0806dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08161e70 con 0x7f7e0815e0b0 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.688+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7f7df8001670 con 0x7f7e0815e0b0 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 >> v1:172.21.15.32:6789/0 conn(0x7f7e0815e0b0 legacy=0x7f7e0806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 0x7f7df4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4003f80 0x7f7df4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4004d90 0x7f7df4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0815e490 con 0x7f7df4004d90 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08161b90 con 0x7f7df4003f80 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df400ba10 con 0x7f7e080b5b60 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4004d90 0x7f7df4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e07fff700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 0x7f7df4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e077fe700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4003f80 0x7f7df4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 -- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 msgr2=0x7f7df4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e080b5b60 0x7f7df4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 -- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4003f80 msgr2=0x7f7df4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4003f80 0x7f7df4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 -- 172.21.15.32:0/3689287330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df40035b0 con 0x7f7df4004d90 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e077fe700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4003f80 0x7f7df4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:13.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4004d90 0x7f7df4005210 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009fc0 tx=0x7f7df4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703aac9377471880 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000ebd0 con 0x7f7df4004d90 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0000f040 con 0x7f7df4004d90 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.689+0000 7f7e06ffd700 1 -- 172.21.15.32:0/3689287330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00014480 con 0x7f7df4004d90 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0e820700 1 -- 172.21.15.32:0/3689287330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4004d90 msgr2=0x7f7df4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0e820700 1 --2- 172.21.15.32:0/3689287330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df4004d90 0x7f7df4005210 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009fc0 tx=0x7f7df4005880 comp rx=0 tx=0).stop 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0cdbd700 1 -- 172.21.15.32:0/3689287330 reap_dead start 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0e820700 1 -- 172.21.15.32:0/3689287330 shutdown_connections 2023-11-24T08:00:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0e820700 1 -- 172.21.15.32:0/3689287330 >> 172.21.15.32:0/3689287330 conn(0x7f7e08071b50 msgr2=0x7f7e08071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0e820700 1 -- 172.21.15.32:0/3689287330 shutdown_connections 2023-11-24T08:00:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.690+0000 7f7e0e820700 1 -- 172.21.15.32:0/3689287330 wait complete. 2023-11-24T08:00:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.691+0000 7f7e0e820700 1 Processor -- start 2023-11-24T08:00:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.691+0000 7f7e0e820700 1 -- start start 2023-11-24T08:00:13.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0e820700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0813e100 0x7f7e08159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0e820700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e0815ead0 0x7f7e0815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0e820700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e0815f490 0x7f7e08140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0e820700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0815a580 con 0x7f7e0813e100 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0e820700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0815f910 con 0x7f7e0815f490 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0e820700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0814ffe0 con 0x7f7e0815ead0 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e07fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0813e100 0x7f7e08159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e07fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0813e100 0x7f7e08159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59146/0 (socket says 172.21.15.32:59146) 2023-11-24T08:00:13.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e07fff700 1 -- 172.21.15.32:0/2407483668 learned_addr learned my addr 172.21.15.32:0/2407483668 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e077fe700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e0815ead0 0x7f7e0815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e07fff700 1 -- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e0815ead0 msgr2=0x7f7e0815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.692+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e0815f490 0x7f7e08140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e07fff700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e0815ead0 0x7f7e0815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e07fff700 1 -- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e0815f490 msgr2=0x7f7e08140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e07fff700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e0815f490 0x7f7e08140850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e07fff700 1 -- 172.21.15.32:0/2407483668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0813e4e0 con 0x7f7e0813e100 2023-11-24T08:00:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e077fe700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e0815ead0 0x7f7e0815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e0815f490 0x7f7e08140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e07fff700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0813e100 0x7f7e08159e70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7df80017a0 tx=0x7f7df80076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df800f8d0 con 0x7f7e0813e100 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df800fa50 con 0x7f7e0813e100 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0814fa20 con 0x7f7e0813e100 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df800fe00 con 0x7f7e0813e100 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.693+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08140d90 con 0x7f7e0813e100 2023-11-24T08:00:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.694+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e08162950 con 0x7f7e0813e100 2023-11-24T08:00:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.694+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7df801d460 con 0x7f7e0813e100 2023-11-24T08:00:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e04ff9700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7de806aed0 0x7f7de806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7df8089f00 con 0x7f7e0813e100 2023-11-24T08:00:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e077fe700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7de806aed0 0x7f7de806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e04ff9700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7f7de8070d30 0x7f7de8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 --> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7de8073820 con 0x7f7de8070d30 2023-11-24T08:00:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7df808a350 con 0x7f7e0813e100 2023-11-24T08:00:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7f7de8070d30 0x7f7de8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e077fe700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7de806aed0 0x7f7de806d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e0815c860 tx=0x7f7df0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.695+0000 7f7e0cdbd700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7f7de8070d30 0x7f7de8073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.696+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== osd.1 v2:172.21.15.32:6800/835572876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7de8073820 con 0x7f7de8070d30 2023-11-24T08:00:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.711+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 --> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7e080b6040 con 0x7f7de8070d30 2023-11-24T08:00:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.711+0000 7f7e04ff9700 1 -- 172.21.15.32:0/2407483668 <== osd.1 v2:172.21.15.32:6800/835572876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7e0000e9d0 con 0x7f7de8070d30 2023-11-24T08:00:13.714 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:13.714 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:13.714 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7f7de8070d30 msgr2=0x7f7de8073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:6800/835572876,v1:172.21.15.32:6801/835572876] conn(0x7f7de8070d30 0x7f7de8073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7de806aed0 msgr2=0x7f7de806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f7de806aed0 0x7f7de806d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e0815c860 tx=0x7f7df0008040 comp rx=0 tx=0).stop 2023-11-24T08:00:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0813e100 msgr2=0x7f7e08159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 --2- 172.21.15.32:0/2407483668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0813e100 0x7f7e08159e70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7df80017a0 tx=0x7f7df80076a0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0cdbd700 1 -- 172.21.15.32:0/2407483668 reap_dead start 2023-11-24T08:00:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 shutdown_connections 2023-11-24T08:00:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 >> 172.21.15.32:0/2407483668 conn(0x7f7e08071b50 msgr2=0x7f7e080b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 shutdown_connections 2023-11-24T08:00:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.712+0000 7f7e0e820700 1 -- 172.21.15.32:0/2407483668 wait complete. 2023-11-24T08:00:13.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T08:00:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.850+0000 7fa9f7628700 1 Processor -- start 2023-11-24T08:00:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.850+0000 7fa9f7628700 1 -- start start 2023-11-24T08:00:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f7628700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9f006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f7628700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00af950 con 0x7fa9f006c540 2023-11-24T08:00:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f7628700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00afc30 con 0x7fa9f01446a0 2023-11-24T08:00:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f7628700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0144a80 con 0x7fa9f00adf00 2023-11-24T08:00:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f5bc5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9f006c540 0x7fa9f00b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60872/0 (socket says 172.21.15.32:60872) 2023-11-24T08:00:13.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f5bc5700 1 -- 172.21.15.32:0/2210262959 learned_addr learned my addr 172.21.15.32:0/2210262959 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9f4bc3700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9f006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116213406 0 0) 0x7fa9f00af950 con 0x7fa9f006c540 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9dc0035b0 con 0x7fa9f006c540 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872988690 0 0) 0x7fa9dc0035b0 con 0x7fa9f006c540 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 msgr2=0x7fa9f006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9f006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 >> v1:172.21.15.123:6789/0 conn(0x7fa9f01446a0 legacy=0x7fa9f00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00aff10 con 0x7fa9f006c540 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.851+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fa9e80019f0 con 0x7fa9f006c540 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 >> v1:172.21.15.32:6789/0 conn(0x7fa9f006c540 legacy=0x7fa9f00b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9dc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc004850 0x7fa9dc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9dc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0144a80 con 0x7fa9dc003a30 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc00dcd0 con 0x7fa9dc004850 2023-11-24T08:00:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc00de90 con 0x7fa9f00adf00 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9dc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f4bc3700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc004850 0x7fa9dc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f5bc5700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9dc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 msgr2=0x7fa9dc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9dc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc004850 msgr2=0x7fa9dc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9dc004850 0x7fa9dc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2210262959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0035b0 con 0x7fa9dc003a30 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f5bc5700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f00adf00 0x7fa9dc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9dc003e50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec007fd0 tx=0x7fa9dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251d688bc7004431 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0052d0 con 0x7fa9dc003a30 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec005470 con 0x7fa9dc003a30 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.852+0000 7fa9e7fff700 1 -- 172.21.15.32:0/2210262959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0038c0 con 0x7fa9dc003a30 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f7628700 1 -- 172.21.15.32:0/2210262959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 msgr2=0x7fa9dc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2210262959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9dc003e50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec007fd0 tx=0x7fa9dc005880 comp rx=0 tx=0).stop 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f5bc5700 1 -- 172.21.15.32:0/2210262959 reap_dead start 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f7628700 1 -- 172.21.15.32:0/2210262959 shutdown_connections 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f7628700 1 -- 172.21.15.32:0/2210262959 >> 172.21.15.32:0/2210262959 conn(0x7fa9f0071b90 msgr2=0x7fa9f0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f7628700 1 -- 172.21.15.32:0/2210262959 shutdown_connections 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.853+0000 7fa9f7628700 1 -- 172.21.15.32:0/2210262959 wait complete. 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.854+0000 7fa9f7628700 1 Processor -- start 2023-11-24T08:00:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.854+0000 7fa9f7628700 1 -- start start 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f7628700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9f01f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f7628700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f01538a0 0x7fa9f00b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f7628700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f015d3c0 0x7fa9f015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f7628700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00ca330 con 0x7fa9dc003a30 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f7628700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c9fe0 con 0x7fa9f015d3c0 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f7628700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f015ddb0 con 0x7fa9f01538a0 2023-11-24T08:00:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9f01f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9f01f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59168/0 (socket says 172.21.15.32:59168) 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2011897298 learned_addr learned my addr 172.21.15.32:0/2011897298 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f01538a0 msgr2=0x7fa9f00b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f01538a0 0x7fa9f00b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f015d3c0 msgr2=0x7fa9f015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f015d3c0 0x7fa9f015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.855+0000 7fa9f53c4700 1 -- 172.21.15.32:0/2011897298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f01f7aa0 con 0x7fa9dc003a30 2023-11-24T08:00:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.856+0000 7fa9f53c4700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9f01f7560 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0047b0 tx=0x7fa9ec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.856+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec007240 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.856+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec0073a0 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.856+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec007730 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.856+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f01f7d40 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.856+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f01f8300 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.857+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f00753f0 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.857+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9ec003680 con 0x7fa9dc003a30 2023-11-24T08:00:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.858+0000 7fa9e5ffb700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa9d406aed0 0x7fa9d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.858+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9ec089e90 con 0x7fa9dc003a30 2023-11-24T08:00:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.858+0000 7fa9f4bc3700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa9d406aed0 0x7fa9d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.859+0000 7fa9f4bc3700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa9d406aed0 0x7fa9d406d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9f015a850 tx=0x7fa9e0006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:13.862+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9ec055350 con 0x7fa9dc003a30 2023-11-24T08:00:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.063+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9f00b04d0 con 0x7fa9d406aed0 2023-11-24T08:00:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.064+0000 7fa9e5ffb700 1 -- 172.21.15.32:0/2011897298 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa9e0011680 con 0x7fa9d406aed0 2023-11-24T08:00:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa9d406aed0 msgr2=0x7fa9d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa9d406aed0 0x7fa9d406d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9f015a850 tx=0x7fa9e0006ca0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 msgr2=0x7fa9f01f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9f01f7560 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0047b0 tx=0x7fa9ec001120 comp rx=0 tx=0).stop 2023-11-24T08:00:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 shutdown_connections 2023-11-24T08:00:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa9d406aed0 0x7fa9d406d390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f015d3c0 0x7fa9f015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9dc003a30 0x7fa9f01f7560 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 --2- 172.21.15.32:0/2011897298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9f01538a0 0x7fa9f00b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 >> 172.21.15.32:0/2011897298 conn(0x7fa9f0071b90 msgr2=0x7fa9f00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 shutdown_connections 2023-11-24T08:00:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.066+0000 7fa9f7628700 1 -- 172.21.15.32:0/2011897298 wait complete. 2023-11-24T08:00:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T08:00:14.080 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T08:00:14.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.208+0000 7fe0935a7700 1 Processor -- start 2023-11-24T08:00:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.208+0000 7fe0935a7700 1 -- start start 2023-11-24T08:00:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe0935a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe08c0cf040 0x7fe08c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe0935a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0702e0 0x7fe08c0d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe0935a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0d01c0 con 0x7fe08c0d1ff0 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe0935a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c070910 con 0x7fe08c0cf040 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe0935a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c070bf0 con 0x7fe08c0702e0 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091343700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe08c0cf040 0x7fe08c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091b44700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe08c0d1ff0 0x7fe08c15f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60892/0 (socket says 172.21.15.32:60892) 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091b44700 1 -- 172.21.15.32:0/3953187128 learned_addr learned my addr 172.21.15.32:0/3953187128 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe090b42700 1 --2- 172.21.15.32:0/3953187128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0702e0 0x7fe08c0d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091343700 1 -- 172.21.15.32:0/3953187128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0702e0 msgr2=0x7fe08c0d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091343700 1 --2- 172.21.15.32:0/3953187128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0702e0 0x7fe08c0d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091343700 1 -- 172.21.15.32:0/3953187128 >> v1:172.21.15.32:6789/0 conn(0x7fe08c0d1ff0 legacy=0x7fe08c15f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091343700 1 -- 172.21.15.32:0/3953187128 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0aebb0 con 0x7fe08c0cf040 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe083fff700 1 -- 172.21.15.32:0/3953187128 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668490125 0 0) 0x7fe08c0d01c0 con 0x7fe08c0d1ff0 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe091343700 1 --2- 172.21.15.32:0/3953187128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0cf040 0x7fe08c06fda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009840 tx=0x7fe07c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81580e4772f9db0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.209+0000 7fe083fff700 1 -- 172.21.15.32:0/3953187128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c011830 con 0x7fe08c0cf040 2023-11-24T08:00:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe083fff700 1 -- 172.21.15.32:0/3953187128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c011990 con 0x7fe08c0cf040 2023-11-24T08:00:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe083fff700 1 -- 172.21.15.32:0/3953187128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c011d20 con 0x7fe08c0cf040 2023-11-24T08:00:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 -- 172.21.15.32:0/3953187128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0cf040 msgr2=0x7fe08c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 --2- 172.21.15.32:0/3953187128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0cf040 0x7fe08c06fda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009840 tx=0x7fe07c009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 -- 172.21.15.32:0/3953187128 shutdown_connections 2023-11-24T08:00:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 --2- 172.21.15.32:0/3953187128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0cf040 0x7fe08c06fda0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 --2- 172.21.15.32:0/3953187128 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0702e0 0x7fe08c0d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 -- 172.21.15.32:0/3953187128 >> 172.21.15.32:0/3953187128 conn(0x7fe08c09aa00 msgr2=0x7fe08c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 -- 172.21.15.32:0/3953187128 shutdown_connections 2023-11-24T08:00:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.210+0000 7fe0935a7700 1 -- 172.21.15.32:0/3953187128 wait complete. 2023-11-24T08:00:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.211+0000 7fe0935a7700 1 Processor -- start 2023-11-24T08:00:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.211+0000 7fe0935a7700 1 -- start start 2023-11-24T08:00:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe0935a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0702e0 0x7fe08c0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe0935a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe08c0cf040 0x7fe08c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe0935a7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0d1ff0 0x7fe08c0b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe0935a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c06d080 con 0x7fe08c0702e0 2023-11-24T08:00:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe0935a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0c9360 con 0x7fe08c0cf040 2023-11-24T08:00:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe0935a7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0c9640 con 0x7fe08c0d1ff0 2023-11-24T08:00:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe091343700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0702e0 0x7fe08c0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe091343700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0702e0 0x7fe08c0c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59182/0 (socket says 172.21.15.32:59182) 2023-11-24T08:00:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.212+0000 7fe091343700 1 -- 172.21.15.32:0/2706818974 learned_addr learned my addr 172.21.15.32:0/2706818974 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe091343700 1 -- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0d1ff0 msgr2=0x7fe08c0b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe091343700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe08c0d1ff0 0x7fe08c0b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe091343700 1 -- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe08c0cf040 msgr2=0x7fe08c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe091343700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe08c0cf040 0x7fe08c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe091343700 1 -- 172.21.15.32:0/2706818974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0094a0 con 0x7fe08c0702e0 2023-11-24T08:00:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe091343700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0702e0 0x7fe08c0c8cd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0090b0 tx=0x7fe07c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0038a0 con 0x7fe08c0702e0 2023-11-24T08:00:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c01f070 con 0x7fe08c0702e0 2023-11-24T08:00:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c0c9920 con 0x7fe08c0702e0 2023-11-24T08:00:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c003c20 con 0x7fe08c0702e0 2023-11-24T08:00:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.213+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c158ab0 con 0x7fe08c0702e0 2023-11-24T08:00:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.214+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe08c0d2440 con 0x7fe08c0702e0 2023-11-24T08:00:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.214+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe07c01a430 con 0x7fe08c0702e0 2023-11-24T08:00:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe081ffb700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe07006aed0 0x7fe07006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe07c021070 con 0x7fe08c0702e0 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe081ffb700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fe070070d30 0x7fe070073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 --> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe070073820 con 0x7fe070070d30 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe07c08b4f0 con 0x7fe08c0702e0 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe091b44700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fe070070d30 0x7fe070073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.215+0000 7fe090b42700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe07006aed0 0x7fe07006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.216+0000 7fe091b44700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fe070070d30 0x7fe070073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.216+0000 7fe090b42700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe07006aed0 0x7fe07006d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe0880097f0 tx=0x7fe088008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.216+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== osd.2 v2:172.21.15.32:6824/3901267389 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe070073820 con 0x7fe070070d30 2023-11-24T08:00:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.231+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 --> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe08c0b03c0 con 0x7fe070070d30 2023-11-24T08:00:14.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe081ffb700 1 -- 172.21.15.32:0/2706818974 <== osd.2 v2:172.21.15.32:6824/3901267389 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe084001d00 con 0x7fe070070d30 2023-11-24T08:00:14.234 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:14.235 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:14.235 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fe070070d30 msgr2=0x7fe070073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:6824/3901267389,v1:172.21.15.32:6825/3901267389] conn(0x7fe070070d30 0x7fe070073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe07006aed0 msgr2=0x7fe07006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe07006aed0 0x7fe07006d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe0880097f0 tx=0x7fe088008040 comp rx=0 tx=0).stop 2023-11-24T08:00:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0702e0 msgr2=0x7fe08c0c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 --2- 172.21.15.32:0/2706818974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0702e0 0x7fe08c0c8cd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0090b0 tx=0x7fe07c004750 comp rx=0 tx=0).stop 2023-11-24T08:00:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe091b44700 1 -- 172.21.15.32:0/2706818974 reap_dead start 2023-11-24T08:00:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 shutdown_connections 2023-11-24T08:00:14.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 >> 172.21.15.32:0/2706818974 conn(0x7fe08c09aa00 msgr2=0x7fe08c15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 shutdown_connections 2023-11-24T08:00:14.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.232+0000 7fe0935a7700 1 -- 172.21.15.32:0/2706818974 wait complete. 2023-11-24T08:00:14.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.370+0000 7f1744259700 1 Processor -- start 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 -- start start 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f173c15dfc0 0x7f173c0d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f173c0b2a70 0x7f173c161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c0d2d10 con 0x7f173c0b2a70 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c0cefd0 con 0x7f173c15dfc0 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.371+0000 7f1744259700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0cf2b0 con 0x7f173c0b2fc0 2023-11-24T08:00:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f173c0b2a70 0x7f173c161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f173c0b2a70 0x7f173c161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59196/0 (socket says 172.21.15.32:59196) 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 -- 172.21.15.32:0/2034981943 learned_addr learned my addr 172.21.15.32:0/2034981943 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17417f4700 1 --2- 172.21.15.32:0/2034981943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f1741ff5700 1 --2- 172.21.15.32:0/2034981943 >> v2:172.21.15.123:3300/0 conn(0x7f173c15dfc0 0x7f173c0d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 -- 172.21.15.32:0/2034981943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 msgr2=0x7f173c0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 --2- 172.21.15.32:0/2034981943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c0d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 -- 172.21.15.32:0/2034981943 >> v2:172.21.15.123:3300/0 conn(0x7f173c15dfc0 msgr2=0x7f173c0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 --2- 172.21.15.32:0/2034981943 >> v2:172.21.15.123:3300/0 conn(0x7f173c15dfc0 0x7f173c0d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 -- 172.21.15.32:0/2034981943 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c0d2fb0 con 0x7f173c0b2a70 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f17427f6700 1 --2- 172.21.15.32:0/2034981943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c0b2a70 0x7f173c161b30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f173c0d2230 tx=0x7f173400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798fb18aab65381e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f1740ff3700 1 -- 172.21.15.32:0/2034981943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400e070 con 0x7f173c0b2a70 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f1740ff3700 1 -- 172.21.15.32:0/2034981943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173400b9c0 con 0x7f173c0b2a70 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.372+0000 7f1740ff3700 1 -- 172.21.15.32:0/2034981943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400bd70 con 0x7f173c0b2a70 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 -- 172.21.15.32:0/2034981943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c0b2a70 msgr2=0x7f173c161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 --2- 172.21.15.32:0/2034981943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c0b2a70 0x7f173c161b30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f173c0d2230 tx=0x7f173400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 -- 172.21.15.32:0/2034981943 shutdown_connections 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 --2- 172.21.15.32:0/2034981943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c0b2a70 0x7f173c161b30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 --2- 172.21.15.32:0/2034981943 >> v2:172.21.15.123:3300/0 conn(0x7f173c15dfc0 0x7f173c0d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 --2- 172.21.15.32:0/2034981943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 -- 172.21.15.32:0/2034981943 >> 172.21.15.32:0/2034981943 conn(0x7f173c06b410 msgr2=0x7f173c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 -- 172.21.15.32:0/2034981943 shutdown_connections 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.373+0000 7f1744259700 1 -- 172.21.15.32:0/2034981943 wait complete. 2023-11-24T08:00:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.375+0000 7f1744259700 1 Processor -- start 2023-11-24T08:00:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.375+0000 7f1744259700 1 -- start start 2023-11-24T08:00:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.375+0000 7f1744259700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f173c0b2a70 0x7f173c0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.375+0000 7f1744259700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1744259700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 0x7f173c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1744259700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ce210 con 0x7f173c15dfc0 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1744259700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0cd600 con 0x7f173c0b2a70 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1744259700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0cd8e0 con 0x7f173c0b2fc0 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 0x7f173c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 0x7f173c15f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59202/0 (socket says 172.21.15.32:59202) 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 -- 172.21.15.32:0/264387947 learned_addr learned my addr 172.21.15.32:0/264387947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17417f4700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1741ff5700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f173c0b2a70 0x7f173c0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 -- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 msgr2=0x7f173c159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 -- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f173c0b2a70 msgr2=0x7f173c0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f173c0b2a70 0x7f173c0cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 -- 172.21.15.32:0/264387947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734009b30 con 0x7f173c15dfc0 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17417f4700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f17427f6700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 0x7f173c15f6a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f173c0b5c40 tx=0x7f1734004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734004250 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17340043d0 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734004780 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c0cdbc0 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.376+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c15a4c0 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.377+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c15a680 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.378+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f173401b430 con 0x7f173c15dfc0 2023-11-24T08:00:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.378+0000 7f172e7fc700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f172006aed0 0x7f172006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.378+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f173408a790 con 0x7f173c15dfc0 2023-11-24T08:00:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.378+0000 7f1741ff5700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f172006aed0 0x7f172006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.379+0000 7f1741ff5700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f172006aed0 0x7f172006d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1738009930 tx=0x7f1738008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.383+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1734055bd0 con 0x7f173c15dfc0 2023-11-24T08:00:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.581+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f173c0c7a90 con 0x7f172006aed0 2023-11-24T08:00:14.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.582+0000 7f172e7fc700 1 -- 172.21.15.32:0/264387947 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f173c0c7a90 con 0x7f172006aed0 2023-11-24T08:00:14.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f172006aed0 msgr2=0x7f172006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f172006aed0 0x7f172006d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1738009930 tx=0x7f1738008040 comp rx=0 tx=0).stop 2023-11-24T08:00:14.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 msgr2=0x7f173c15f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 0x7f173c15f6a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f173c0b5c40 tx=0x7f1734004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 shutdown_connections 2023-11-24T08:00:14.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f172006aed0 0x7f172006d390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173c15dfc0 0x7f173c15f6a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f173c0b2a70 0x7f173c0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 --2- 172.21.15.32:0/264387947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f173c0b2fc0 0x7f173c159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 >> 172.21.15.32:0/264387947 conn(0x7f173c06b410 msgr2=0x7f173c0b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 shutdown_connections 2023-11-24T08:00:14.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.584+0000 7f1744259700 1 -- 172.21.15.32:0/264387947 wait complete. 2023-11-24T08:00:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T08:00:14.598 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T08:00:14.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.725+0000 7f9685be6700 1 Processor -- start 2023-11-24T08:00:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 -- start start 2023-11-24T08:00:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b1890 0x7f968015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96800af730 0x7f96800622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96800726c0 0x7f9680072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96800ce970 con 0x7f96800726c0 2023-11-24T08:00:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96800cd4f0 con 0x7f96800af730 2023-11-24T08:00:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f9685be6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b0e10 con 0x7f96800b1890 2023-11-24T08:00:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f967ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96800726c0 0x7f9680072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.726+0000 7f967ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96800726c0 0x7f9680072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59210/0 (socket says 172.21.15.32:59210) 2023-11-24T08:00:14.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 -- 172.21.15.32:0/2789019130 learned_addr learned my addr 172.21.15.32:0/2789019130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:14.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967f7fe700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b1890 0x7f968015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967effd700 1 --2- 172.21.15.32:0/2789019130 >> v2:172.21.15.123:3300/0 conn(0x7f96800af730 0x7f96800622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 -- 172.21.15.32:0/2789019130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b1890 msgr2=0x7f968015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b1890 0x7f968015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 -- 172.21.15.32:0/2789019130 >> v2:172.21.15.123:3300/0 conn(0x7f96800af730 msgr2=0x7f96800622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 --2- 172.21.15.32:0/2789019130 >> v2:172.21.15.123:3300/0 conn(0x7f96800af730 0x7f96800622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 -- 172.21.15.32:0/2789019130 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800ce160 con 0x7f96800726c0 2023-11-24T08:00:14.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967effd700 1 --2- 172.21.15.32:0/2789019130 >> v2:172.21.15.123:3300/0 conn(0x7f96800af730 0x7f96800622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:14.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967f7fe700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b1890 0x7f968015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:14.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967ffff700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f9680072b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9680073730 tx=0x7f967800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a489f9d5aabd2e2e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967e7fc700 1 -- 172.21.15.32:0/2789019130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967800e070 con 0x7f96800726c0 2023-11-24T08:00:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967e7fc700 1 -- 172.21.15.32:0/2789019130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967800b9c0 con 0x7f96800726c0 2023-11-24T08:00:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.727+0000 7f967e7fc700 1 -- 172.21.15.32:0/2789019130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967800bd70 con 0x7f96800726c0 2023-11-24T08:00:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 -- 172.21.15.32:0/2789019130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 msgr2=0x7f9680072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f9680072b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9680073730 tx=0x7f967800b330 comp rx=0 tx=0).stop 2023-11-24T08:00:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 -- 172.21.15.32:0/2789019130 shutdown_connections 2023-11-24T08:00:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f9680072b50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 --2- 172.21.15.32:0/2789019130 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b1890 0x7f968015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 --2- 172.21.15.32:0/2789019130 >> v2:172.21.15.123:3300/0 conn(0x7f96800af730 0x7f96800622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 -- 172.21.15.32:0/2789019130 >> 172.21.15.32:0/2789019130 conn(0x7f96800b1e90 msgr2=0x7f96800b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 -- 172.21.15.32:0/2789019130 shutdown_connections 2023-11-24T08:00:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.728+0000 7f9685be6700 1 -- 172.21.15.32:0/2789019130 wait complete. 2023-11-24T08:00:14.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.729+0000 7f9685be6700 1 Processor -- start 2023-11-24T08:00:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.729+0000 7f9685be6700 1 -- start start 2023-11-24T08:00:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f9685be6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f968014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f9685be6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96800af730 0x7f9680152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f9685be6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b5bb0 0x7f96801590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f9685be6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9680151270 con 0x7f96800726c0 2023-11-24T08:00:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f9685be6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b3e90 con 0x7f96800af730 2023-11-24T08:00:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f9685be6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b4170 con 0x7f96800b5bb0 2023-11-24T08:00:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f968014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f968014d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59212/0 (socket says 172.21.15.32:59212) 2023-11-24T08:00:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 -- 172.21.15.32:0/43170268 learned_addr learned my addr 172.21.15.32:0/43170268 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967ffff700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b5bb0 0x7f96801590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967effd700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96800af730 0x7f9680152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 -- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b5bb0 msgr2=0x7f96801590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96800b5bb0 0x7f96801590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 -- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96800af730 msgr2=0x7f9680152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96800af730 0x7f9680152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967f7fe700 1 -- 172.21.15.32:0/43170268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009b30 con 0x7f96800726c0 2023-11-24T08:00:14.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.730+0000 7f967effd700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96800af730 0x7f9680152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:14.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f967f7fe700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f968014d740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f967000c7f0 tx=0x7f967000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011800 con 0x7f96800726c0 2023-11-24T08:00:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670011960 con 0x7f96800726c0 2023-11-24T08:00:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96800b4450 con 0x7f96800726c0 2023-11-24T08:00:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011d10 con 0x7f96800726c0 2023-11-24T08:00:14.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9680152cb0 con 0x7f96800726c0 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.731+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96800ce690 con 0x7f96800726c0 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.732+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9670018360 con 0x7f96800726c0 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f9677fff700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f966006aed0 0x7f966006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9670089200 con 0x7f96800726c0 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f9677fff700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f9660070d30 0x7f9660073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 --> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9660073820 con 0x7f9660070d30 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f967effd700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f966006aed0 0x7f966006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967008b8c0 con 0x7f96800726c0 2023-11-24T08:00:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f967ffff700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f9660070d30 0x7f9660073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f967ffff700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f9660070d30 0x7f9660073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.733+0000 7f967effd700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f966006aed0 0x7f966006d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96800b1360 tx=0x7f9668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.734+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== osd.3 v2:172.21.15.32:6808/2834432589 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9660073820 con 0x7f9660070d30 2023-11-24T08:00:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.749+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 --> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f96800cdc20 con 0x7f9660070d30 2023-11-24T08:00:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.749+0000 7f9677fff700 1 -- 172.21.15.32:0/43170268 <== osd.3 v2:172.21.15.32:6808/2834432589 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9678003a20 con 0x7f9660070d30 2023-11-24T08:00:14.754 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:14.754 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:14.755 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.749+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f9660070d30 msgr2=0x7f9660073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.749+0000 7f9685be6700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:6808/2834432589,v1:172.21.15.32:6809/2834432589] conn(0x7f9660070d30 0x7f9660073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f966006aed0 msgr2=0x7f966006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f966006aed0 0x7f966006d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f96800b1360 tx=0x7f9668008040 comp rx=0 tx=0).stop 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 msgr2=0x7f968014d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 --2- 172.21.15.32:0/43170268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800726c0 0x7f968014d740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f967000c7f0 tx=0x7f967000cbb0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f967ffff700 1 -- 172.21.15.32:0/43170268 reap_dead start 2023-11-24T08:00:14.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 shutdown_connections 2023-11-24T08:00:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 >> 172.21.15.32:0/43170268 conn(0x7f96800b1e90 msgr2=0x7f9680061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 shutdown_connections 2023-11-24T08:00:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.750+0000 7f9685be6700 1 -- 172.21.15.32:0/43170268 wait complete. 2023-11-24T08:00:14.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T08:00:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.888+0000 7f64db803700 1 Processor -- start 2023-11-24T08:00:14.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64db803700 1 -- start start 2023-11-24T08:00:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64db803700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64d40b0fb0 0x7f64d40cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64db803700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64d40b1390 0x7f64d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64db803700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40b1c60 con 0x7f64d40d0280 2023-11-24T08:00:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64db803700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d40d0720 con 0x7f64d40b1390 2023-11-24T08:00:14.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64db803700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d40d0a00 con 0x7f64d40b0fb0 2023-11-24T08:00:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64d8d9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64d40b1390 0x7f64d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.889+0000 7f64d9da0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64d40d0280 0x7f64d415d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60938/0 (socket says 172.21.15.32:60938) 2023-11-24T08:00:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d9da0700 1 -- 172.21.15.32:0/1921116179 learned_addr learned my addr 172.21.15.32:0/1921116179 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d959f700 1 --2- 172.21.15.32:0/1921116179 >> v2:172.21.15.123:3300/0 conn(0x7f64d40b0fb0 0x7f64d40cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64c7fff700 1 -- 172.21.15.32:0/1921116179 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422219727 0 0) 0x7f64d40b1c60 con 0x7f64d40d0280 2023-11-24T08:00:14.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64c7fff700 1 -- 172.21.15.32:0/1921116179 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c00035b0 con 0x7f64d40d0280 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d8d9e700 1 -- 172.21.15.32:0/1921116179 >> v2:172.21.15.123:3300/0 conn(0x7f64d40b0fb0 msgr2=0x7f64d40cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d8d9e700 1 --2- 172.21.15.32:0/1921116179 >> v2:172.21.15.123:3300/0 conn(0x7f64d40b0fb0 0x7f64d40cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d8d9e700 1 -- 172.21.15.32:0/1921116179 >> v1:172.21.15.32:6789/0 conn(0x7f64d40d0280 legacy=0x7f64d415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d8d9e700 1 -- 172.21.15.32:0/1921116179 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40d0ce0 con 0x7f64d40b1390 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64d8d9e700 1 --2- 172.21.15.32:0/1921116179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d406b750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f64d406df90 tx=0x7f64d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced7df8ec1cfef59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64c7fff700 1 -- 172.21.15.32:0/1921116179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0011830 con 0x7f64d40b1390 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64c7fff700 1 -- 172.21.15.32:0/1921116179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d0011990 con 0x7f64d40b1390 2023-11-24T08:00:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.890+0000 7f64c7fff700 1 -- 172.21.15.32:0/1921116179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0011d20 con 0x7f64d40b1390 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 -- 172.21.15.32:0/1921116179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 msgr2=0x7f64d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 --2- 172.21.15.32:0/1921116179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d406b750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f64d406df90 tx=0x7f64d0009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 -- 172.21.15.32:0/1921116179 shutdown_connections 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 --2- 172.21.15.32:0/1921116179 >> v2:172.21.15.123:3300/0 conn(0x7f64d40b0fb0 0x7f64d40cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 --2- 172.21.15.32:0/1921116179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d406b750 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 -- 172.21.15.32:0/1921116179 >> 172.21.15.32:0/1921116179 conn(0x7f64d409aa00 msgr2=0x7f64d409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 -- 172.21.15.32:0/1921116179 shutdown_connections 2023-11-24T08:00:14.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.891+0000 7f64db803700 1 -- 172.21.15.32:0/1921116179 wait complete. 2023-11-24T08:00:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.892+0000 7f64db803700 1 Processor -- start 2023-11-24T08:00:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.892+0000 7f64db803700 1 -- start start 2023-11-24T08:00:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64db803700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64d40b0fb0 0x7f64d414e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64db803700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d40cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64db803700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40d0280 0x7f64d40ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64db803700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b67c0 con 0x7f64d40b1390 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64db803700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b5720 con 0x7f64d40d0280 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64db803700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b5a00 con 0x7f64d40b0fb0 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d40cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d40cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59230/0 (socket says 172.21.15.32:59230) 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 -- 172.21.15.32:0/2353057070 learned_addr learned my addr 172.21.15.32:0/2353057070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 -- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64d40b0fb0 msgr2=0x7f64d414e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d959f700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64d40b0fb0 0x7f64d414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d9da0700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40d0280 0x7f64d40ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64d40b0fb0 0x7f64d414e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 -- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40d0280 msgr2=0x7f64d40ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40d0280 0x7f64d40ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d8d9e700 1 -- 172.21.15.32:0/2353057070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00094a0 con 0x7f64d40b1390 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d959f700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64d40b0fb0 0x7f64d414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.893+0000 7f64d9da0700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40d0280 0x7f64d40ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.894+0000 7f64d8d9e700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d40cc600 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009b60 tx=0x7f64d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.894+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d00036a0 con 0x7f64d40b1390 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.894+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d001f070 con 0x7f64d40b1390 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.894+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d40b5ce0 con 0x7f64d40b1390 2023-11-24T08:00:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.894+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0003a00 con 0x7f64d40b1390 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.894+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d40b62a0 con 0x7f64d40b1390 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.895+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d40b6610 con 0x7f64d40b1390 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.895+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64d0003b60 con 0x7f64d40b1390 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.896+0000 7f64c5ffb700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f64b806aed0 0x7f64b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.896+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f64d0021070 con 0x7f64d40b1390 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.896+0000 7f64d959f700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f64b806aed0 0x7f64b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.897+0000 7f64d959f700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f64b806aed0 0x7f64b806d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009a00 tx=0x7f64c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:14.900+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64d0056020 con 0x7f64d40b1390 2023-11-24T08:00:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.100+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d40d2390 con 0x7f64b806aed0 2023-11-24T08:00:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.101+0000 7f64c5ffb700 1 -- 172.21.15.32:0/2353057070 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f64c8009610 con 0x7f64b806aed0 2023-11-24T08:00:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f64b806aed0 msgr2=0x7f64b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f64b806aed0 0x7f64b806d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009a00 tx=0x7f64c8008040 comp rx=0 tx=0).stop 2023-11-24T08:00:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 msgr2=0x7f64d40cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d40cc600 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009b60 tx=0x7f64d0004750 comp rx=0 tx=0).stop 2023-11-24T08:00:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 shutdown_connections 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f64b806aed0 0x7f64b806d390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40d0280 0x7f64d40ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f64d40b0fb0 0x7f64d414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 --2- 172.21.15.32:0/2353057070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64d40b1390 0x7f64d40cc600 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 >> 172.21.15.32:0/2353057070 conn(0x7f64d409aa00 msgr2=0x7f64d4160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 shutdown_connections 2023-11-24T08:00:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.103+0000 7f64db803700 1 -- 172.21.15.32:0/2353057070 wait complete. 2023-11-24T08:00:15.107 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T08:00:15.116 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T08:00:15.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.244+0000 7fb8ae7ea700 1 Processor -- start 2023-11-24T08:00:15.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.244+0000 7fb8ae7ea700 1 -- start start 2023-11-24T08:00:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.244+0000 7fb8ae7ea700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8a80cf040 0x7fb8a806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8ae7ea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80d01c0 con 0x7fb8a80702e0 2023-11-24T08:00:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8ae7ea700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a8070910 con 0x7fb8a80d21c0 2023-11-24T08:00:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8ae7ea700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a8070bf0 con 0x7fb8a80cf040 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8a80702e0 0x7fb8a80d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60948/0 (socket says 172.21.15.32:60948) 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a77fe700 1 -- 172.21.15.32:0/3345020386 learned_addr learned my addr 172.21.15.32:0/3345020386 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a7fff700 1 --2- 172.21.15.32:0/3345020386 >> v2:172.21.15.123:3300/0 conn(0x7fb8a80cf040 0x7fb8a806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660194 0 0) 0x7fb8a80d01c0 con 0x7fb8a80702e0 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8900035b0 con 0x7fb8a80702e0 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029995122 0 0) 0x7fb8900035b0 con 0x7fb8a80702e0 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 >> v2:172.21.15.123:3300/0 conn(0x7fb8a80cf040 msgr2=0x7fb8a806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 --2- 172.21.15.32:0/3345020386 >> v2:172.21.15.123:3300/0 conn(0x7fb8a80cf040 0x7fb8a806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 >> v1:172.21.15.123:6789/0 conn(0x7fb8a80d21c0 legacy=0x7fb8a80af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80aebb0 con 0x7fb8a80702e0 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 597153580 0 0) 0x7fb89c001670 con 0x7fb8a80702e0 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 >> v1:172.21.15.32:6789/0 conn(0x7fb8a80702e0 legacy=0x7fb8a80d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb890003a00 0x7fb890003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb890004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb890005210 0x7fb89000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8070bf0 con 0x7fb890004850 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89000c0b0 con 0x7fb890005210 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89000c270 con 0x7fb890003a00 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.245+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb890004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a7fff700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb890003a00 0x7fb890003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8acd87700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb890005210 0x7fb89000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a77fe700 1 -- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb890003a00 msgr2=0x7fb890003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb890003a00 0x7fb890003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a77fe700 1 -- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb890005210 msgr2=0x7fb89000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb890005210 0x7fb89000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8acd87700 1 -- 172.21.15.32:0/3345020386 reap_dead start 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a77fe700 1 -- 172.21.15.32:0/3345020386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900035b0 con 0x7fb890004850 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb890004cd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c240 tx=0x7fb890005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca6a390ede0d6bb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c007c10 con 0x7fb890004850 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c00cde0 con 0x7fb890004850 2023-11-24T08:00:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.246+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/3345020386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c005ba0 con 0x7fb890004850 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/3345020386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 msgr2=0x7fb890004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb890004cd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c240 tx=0x7fb890005880 comp rx=0 tx=0).stop 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/3345020386 shutdown_connections 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 --2- 172.21.15.32:0/3345020386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb890004cd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/3345020386 >> 172.21.15.32:0/3345020386 conn(0x7fb8a809aa00 msgr2=0x7fb8a809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/3345020386 shutdown_connections 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.247+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/3345020386 wait complete. 2023-11-24T08:00:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.248+0000 7fb8ae7ea700 1 Processor -- start 2023-11-24T08:00:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.248+0000 7fb8ae7ea700 1 -- start start 2023-11-24T08:00:15.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8ae7ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb8a806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8ae7ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a806c7b0 0x7fb8a80b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8ae7ea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb8a8153c90 0x7fb8a8154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8ae7ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8154680 con 0x7fb890004850 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8ae7ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80c8e70 con 0x7fb8a806c7b0 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8ae7ea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80c9150 con 0x7fb8a8153c90 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb8a806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb8a806b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59248/0 (socket says 172.21.15.32:59248) 2023-11-24T08:00:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 -- 172.21.15.32:0/1314973242 learned_addr learned my addr 172.21.15.32:0/1314973242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a806c7b0 0x7fb8a80b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8acd87700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb8a8153c90 0x7fb8a8154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 -- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb8a8153c90 msgr2=0x7fb8a8154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb8a8153c90 0x7fb8a8154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 -- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a806c7b0 msgr2=0x7fb8a80b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8a806c7b0 0x7fb8a80b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 -- 172.21.15.32:0/1314973242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80c9430 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.249+0000 7fb8a7fff700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb8a806b9b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb898011aa0 tx=0x7fb898011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.250+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89800f940 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.250+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89800faa0 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.250+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a80c9710 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.250+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89800fe50 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.250+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a806bef0 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.250+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8a80cf420 con 0x7fb890004850 2023-11-24T08:00:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.251+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb89801d400 con 0x7fb890004850 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.251+0000 7fb8a4ff9700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb89406aed0 0x7fb89406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb898089ff0 con 0x7fb890004850 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb89406aed0 0x7fb89406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8a4ff9700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7fb894070d30 0x7fb894073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 --> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb894073820 con 0x7fb894070d30 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb89808a440 con 0x7fb890004850 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8acd87700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7fb894070d30 0x7fb894073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb89406aed0 0x7fb89406d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb89c001930 tx=0x7fb89c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.252+0000 7fb8acd87700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7fb894070d30 0x7fb894073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.253+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== osd.4 v2:172.21.15.123:6808/1395851986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb894073820 con 0x7fb894070d30 2023-11-24T08:00:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.268+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 --> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb8a80c9d60 con 0x7fb894070d30 2023-11-24T08:00:15.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8a4ff9700 1 -- 172.21.15.32:0/1314973242 <== osd.4 v2:172.21.15.123:6808/1395851986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb8a80c9d60 con 0x7fb894070d30 2023-11-24T08:00:15.271 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:15.271 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:15.271 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7fb894070d30 msgr2=0x7fb894073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6808/1395851986,v1:172.21.15.123:6809/1395851986] conn(0x7fb894070d30 0x7fb894073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb89406aed0 msgr2=0x7fb89406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fb89406aed0 0x7fb89406d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb89c001930 tx=0x7fb89c007000 comp rx=0 tx=0).stop 2023-11-24T08:00:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 msgr2=0x7fb8a806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 --2- 172.21.15.32:0/1314973242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890004850 0x7fb8a806b9b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb898011aa0 tx=0x7fb898011db0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8acd87700 1 -- 172.21.15.32:0/1314973242 reap_dead start 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 shutdown_connections 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 >> 172.21.15.32:0/1314973242 conn(0x7fb8a809aa00 msgr2=0x7fb8a80af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 shutdown_connections 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.269+0000 7fb8ae7ea700 1 -- 172.21.15.32:0/1314973242 wait complete. 2023-11-24T08:00:15.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T08:00:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.408+0000 7f02f6285700 1 Processor -- start 2023-11-24T08:00:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.408+0000 7f02f6285700 1 -- start start 2023-11-24T08:00:15.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02f6285700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f02f015bdd0 0x7f02f01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02f6285700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f00b3370 0x7f02f00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02f6285700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f00b2030 con 0x7f02f00d1c70 2023-11-24T08:00:15.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02f6285700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f00b2f50 con 0x7f02f00b3370 2023-11-24T08:00:15.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02f6285700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f00b22d0 con 0x7f02f015bdd0 2023-11-24T08:00:15.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f00b3370 0x7f02f00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f00b3370 0x7f02f00b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59260/0 (socket says 172.21.15.32:59260) 2023-11-24T08:00:15.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 -- 172.21.15.32:0/467870877 learned_addr learned my addr 172.21.15.32:0/467870877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02effff700 1 --2- 172.21.15.32:0/467870877 >> v2:172.21.15.123:3300/0 conn(0x7f02f015bdd0 0x7f02f01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 -- 172.21.15.32:0/467870877 >> v2:172.21.15.123:3300/0 conn(0x7f02f015bdd0 msgr2=0x7f02f01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/467870877 >> v2:172.21.15.123:3300/0 conn(0x7f02f015bdd0 0x7f02f01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 -- 172.21.15.32:0/467870877 >> v1:172.21.15.123:6789/0 conn(0x7f02f00d1c70 legacy=0x7f02f00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 -- 172.21.15.32:0/467870877 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00b25b0 con 0x7f02f00b3370 2023-11-24T08:00:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.409+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/467870877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f00b1af0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f02f006ec10 tx=0x7f02e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b0766bb99ce1d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02eeffd700 1 -- 172.21.15.32:0/467870877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e000b870 con 0x7f02f00b3370 2023-11-24T08:00:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02eeffd700 1 -- 172.21.15.32:0/467870877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e000b9d0 con 0x7f02f00b3370 2023-11-24T08:00:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02eeffd700 1 -- 172.21.15.32:0/467870877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e000bd60 con 0x7f02f00b3370 2023-11-24T08:00:15.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 -- 172.21.15.32:0/467870877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 msgr2=0x7f02f00b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 --2- 172.21.15.32:0/467870877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f00b1af0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f02f006ec10 tx=0x7f02e0009bd0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 -- 172.21.15.32:0/467870877 shutdown_connections 2023-11-24T08:00:15.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 --2- 172.21.15.32:0/467870877 >> v2:172.21.15.123:3300/0 conn(0x7f02f015bdd0 0x7f02f01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 --2- 172.21.15.32:0/467870877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f00b1af0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 -- 172.21.15.32:0/467870877 >> 172.21.15.32:0/467870877 conn(0x7f02f00731a0 msgr2=0x7f02f009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 -- 172.21.15.32:0/467870877 shutdown_connections 2023-11-24T08:00:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.410+0000 7f02f6285700 1 -- 172.21.15.32:0/467870877 wait complete. 2023-11-24T08:00:15.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.411+0000 7f02f6285700 1 Processor -- start 2023-11-24T08:00:15.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 -- start start 2023-11-24T08:00:15.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f01570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02f00d1c70 0x7f02f00b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02f015bdd0 0x7f02f00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f006d0b0 con 0x7f02f00b3370 2023-11-24T08:00:15.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f01575e0 con 0x7f02f015bdd0 2023-11-24T08:00:15.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02f6285700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f006c810 con 0x7f02f00d1c70 2023-11-24T08:00:15.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f01570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f01570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59264/0 (socket says 172.21.15.32:59264) 2023-11-24T08:00:15.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.412+0000 7f02effff700 1 -- 172.21.15.32:0/488822745 learned_addr learned my addr 172.21.15.32:0/488822745 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:15.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02f00d1c70 0x7f02f00b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02f4822700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02f015bdd0 0x7f02f00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02effff700 1 -- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02f00d1c70 msgr2=0x7f02f00b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02effff700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02f00d1c70 0x7f02f00b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02effff700 1 -- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02f015bdd0 msgr2=0x7f02f00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02effff700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02f015bdd0 0x7f02f00b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02effff700 1 -- 172.21.15.32:0/488822745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0009550 con 0x7f02f00b3370 2023-11-24T08:00:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02f4822700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02f015bdd0 0x7f02f00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:15.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02effff700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f01570a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02d800d800 tx=0x7f02d800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d80125f0 con 0x7f02f00b3370 2023-11-24T08:00:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d8012750 con 0x7f02f00b3370 2023-11-24T08:00:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f006caf0 con 0x7f02f00b3370 2023-11-24T08:00:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8012b00 con 0x7f02f00b3370 2023-11-24T08:00:15.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.413+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f00c8f50 con 0x7f02f00b3370 2023-11-24T08:00:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.414+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f00c8db0 con 0x7f02f00b3370 2023-11-24T08:00:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.415+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02d8012c60 con 0x7f02f00b3370 2023-11-24T08:00:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.415+0000 7f02ecff9700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f02d006aed0 0x7f02d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.415+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f02d80895f0 con 0x7f02f00b3370 2023-11-24T08:00:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.415+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f02d006aed0 0x7f02d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.416+0000 7f02ef7fe700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f02d006aed0 0x7f02d006d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f02f00b37e0 tx=0x7f02e0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.419+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02f00c8db0 con 0x7f02f00b3370 2023-11-24T08:00:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.621+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f02f00b2b70 con 0x7f02d006aed0 2023-11-24T08:00:15.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.622+0000 7f02ecff9700 1 -- 172.21.15.32:0/488822745 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f02e000bb40 con 0x7f02d006aed0 2023-11-24T08:00:15.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f02d006aed0 msgr2=0x7f02d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f02d006aed0 0x7f02d006d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f02f00b37e0 tx=0x7f02e0008030 comp rx=0 tx=0).stop 2023-11-24T08:00:15.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 msgr2=0x7f02f01570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f01570a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02d800d800 tx=0x7f02d800db10 comp rx=0 tx=0).stop 2023-11-24T08:00:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 shutdown_connections 2023-11-24T08:00:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f02d006aed0 0x7f02d006d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02f015bdd0 0x7f02f00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f00b3370 0x7f02f01570a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 --2- 172.21.15.32:0/488822745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02f00d1c70 0x7f02f00b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 >> 172.21.15.32:0/488822745 conn(0x7f02f00731a0 msgr2=0x7f02f015f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 shutdown_connections 2023-11-24T08:00:15.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.624+0000 7f02f6285700 1 -- 172.21.15.32:0/488822745 wait complete. 2023-11-24T08:00:15.628 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T08:00:15.638 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T08:00:15.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.766+0000 7f875ef9f700 1 Processor -- start 2023-11-24T08:00:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.766+0000 7f875ef9f700 1 -- start start 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875ef9f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f875815ee20 0x7f875806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875ef9f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f875806fa00 0x7f87580b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875ef9f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875806fe60 con 0x7f87580b7350 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875ef9f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87580b77e0 con 0x7f875806fa00 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875ef9f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87581621a0 con 0x7f875815ee20 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f875806fa00 0x7f87580b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f875806fa00 0x7f87580b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59272/0 (socket says 172.21.15.32:59272) 2023-11-24T08:00:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 -- 172.21.15.32:0/4039211365 learned_addr learned my addr 172.21.15.32:0/4039211365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875df9d700 1 --2- 172.21.15.32:0/4039211365 >> v2:172.21.15.123:3300/0 conn(0x7f875815ee20 0x7f875806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 -- 172.21.15.32:0/4039211365 >> v2:172.21.15.123:3300/0 conn(0x7f875815ee20 msgr2=0x7f875806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 --2- 172.21.15.32:0/4039211365 >> v2:172.21.15.123:3300/0 conn(0x7f875815ee20 0x7f875806f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 -- 172.21.15.32:0/4039211365 >> v1:172.21.15.123:6789/0 conn(0x7f87580b7350 legacy=0x7f87580d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 -- 172.21.15.32:0/4039211365 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758161890 con 0x7f875806fa00 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875d79c700 1 --2- 172.21.15.32:0/4039211365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580b6e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8758070d80 tx=0x7f8748009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c6d3dba7e6a2ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875cf9b700 1 -- 172.21.15.32:0/4039211365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800b870 con 0x7f875806fa00 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875cf9b700 1 -- 172.21.15.32:0/4039211365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874800b9d0 con 0x7f875806fa00 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.767+0000 7f875cf9b700 1 -- 172.21.15.32:0/4039211365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800bd60 con 0x7f875806fa00 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 -- 172.21.15.32:0/4039211365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 msgr2=0x7f87580b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 --2- 172.21.15.32:0/4039211365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580b6e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8758070d80 tx=0x7f8748009bd0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 -- 172.21.15.32:0/4039211365 shutdown_connections 2023-11-24T08:00:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 --2- 172.21.15.32:0/4039211365 >> v2:172.21.15.123:3300/0 conn(0x7f875815ee20 0x7f875806f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 --2- 172.21.15.32:0/4039211365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580b6e10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 -- 172.21.15.32:0/4039211365 >> 172.21.15.32:0/4039211365 conn(0x7f875809aa00 msgr2=0x7f875809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 -- 172.21.15.32:0/4039211365 shutdown_connections 2023-11-24T08:00:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.768+0000 7f875ef9f700 1 -- 172.21.15.32:0/4039211365 wait complete. 2023-11-24T08:00:15.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 Processor -- start 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 -- start start 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87580b7350 0x7f87580cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875815ee20 0x7f87581e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87580cb0b0 con 0x7f875806fa00 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87580cb390 con 0x7f875815ee20 2023-11-24T08:00:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875ef9f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580cb670 con 0x7f87580b7350 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875df9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875df9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59278/0 (socket says 172.21.15.32:59278) 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.770+0000 7f875df9d700 1 -- 172.21.15.32:0/2994553200 learned_addr learned my addr 172.21.15.32:0/2994553200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875e79e700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875815ee20 0x7f87581e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875d79c700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87580b7350 0x7f87580cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875df9d700 1 -- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87580b7350 msgr2=0x7f87580cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875df9d700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87580b7350 0x7f87580cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875df9d700 1 -- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875815ee20 msgr2=0x7f87581e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875df9d700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875815ee20 0x7f87581e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875df9d700 1 -- 172.21.15.32:0/2994553200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8748009550 con 0x7f875806fa00 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875e79e700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f875815ee20 0x7f87581e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875d79c700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87580b7350 0x7f87580cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875df9d700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580cc430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f875400d800 tx=0x7f875400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540125f0 con 0x7f875806fa00 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754012750 con 0x7f875806fa00 2023-11-24T08:00:15.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87580cb950 con 0x7f875806fa00 2023-11-24T08:00:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754012b00 con 0x7f875806fa00 2023-11-24T08:00:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.771+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87580cbf10 con 0x7f875806fa00 2023-11-24T08:00:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.772+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87580b7000 con 0x7f875806fa00 2023-11-24T08:00:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8754012c60 con 0x7f875806fa00 2023-11-24T08:00:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f874e7fc700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f873c06aed0 0x7f873c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8754089610 con 0x7f875806fa00 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f874e7fc700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f873c070d30 0x7f873c073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 --> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f873c073820 con 0x7f873c070d30 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f875408bcd0 con 0x7f875806fa00 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f875d79c700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f873c06aed0 0x7f873c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.773+0000 7f875e79e700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f873c070d30 0x7f873c073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.774+0000 7f875e79e700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f873c070d30 0x7f873c073150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.774+0000 7f875d79c700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f873c06aed0 0x7f873c06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8748000c00 tx=0x7f8748008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.775+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== osd.5 v2:172.21.15.123:6820/3666097374 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f873c073820 con 0x7f873c070d30 2023-11-24T08:00:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 --> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8758161b50 con 0x7f873c070d30 2023-11-24T08:00:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f874e7fc700 1 -- 172.21.15.32:0/2994553200 <== osd.5 v2:172.21.15.123:6820/3666097374 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8750003a00 con 0x7f873c070d30 2023-11-24T08:00:15.792 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:15.792 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:15.792 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f873c070d30 msgr2=0x7f873c073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6820/3666097374,v1:172.21.15.123:6822/3666097374] conn(0x7f873c070d30 0x7f873c073150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f873c06aed0 msgr2=0x7f873c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f873c06aed0 0x7f873c06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8748000c00 tx=0x7f8748008030 comp rx=0 tx=0).stop 2023-11-24T08:00:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 msgr2=0x7f87580cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 --2- 172.21.15.32:0/2994553200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f875806fa00 0x7f87580cc430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f875400d800 tx=0x7f875400dbc0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875e79e700 1 -- 172.21.15.32:0/2994553200 reap_dead start 2023-11-24T08:00:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 shutdown_connections 2023-11-24T08:00:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.790+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 >> 172.21.15.32:0/2994553200 conn(0x7f875809aa00 msgr2=0x7f87580b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.791+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 shutdown_connections 2023-11-24T08:00:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.791+0000 7f875ef9f700 1 -- 172.21.15.32:0/2994553200 wait complete. 2023-11-24T08:00:15.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T08:00:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.928+0000 7f67f459c700 1 Processor -- start 2023-11-24T08:00:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.929+0000 7f67f459c700 1 -- start start 2023-11-24T08:00:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.929+0000 7f67f459c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67ec15ae20 0x7f67ec0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.929+0000 7f67f459c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0cfd40 con 0x7f67ec15b200 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.929+0000 7f67f459c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0affa0 con 0x7f67ec0b1a70 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.929+0000 7f67f459c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ec0d0020 con 0x7f67ec15ae20 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f2338700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67ec15ae20 0x7f67ec0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f1b37700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67ec15b200 0x7f67ec0b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32768/0 (socket says 172.21.15.32:32768) 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f2338700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67ec15ae20 0x7f67ec0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59288/0 (socket says 172.21.15.32:59288) 2023-11-24T08:00:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f1b37700 1 -- 172.21.15.32:0/2370154367 learned_addr learned my addr 172.21.15.32:0/2370154367 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f2338700 1 -- 172.21.15.32:0/2370154367 >> v1:172.21.15.123:6789/0 conn(0x7f67ec0b1a70 legacy=0x7f67ec164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f2338700 1 -- 172.21.15.32:0/2370154367 >> v1:172.21.15.32:6789/0 conn(0x7f67ec15b200 legacy=0x7f67ec0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f2338700 1 -- 172.21.15.32:0/2370154367 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0b0280 con 0x7f67ec15ae20 2023-11-24T08:00:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f1336700 1 -- 172.21.15.32:0/2370154367 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106220834 0 0) 0x7f67ec0cfd40 con 0x7f67ec15b200 2023-11-24T08:00:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f2338700 1 --2- 172.21.15.32:0/2370154367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15ae20 0x7f67ec0b29a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009840 tx=0x7f67e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d40d96ba4017ce1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f1336700 1 -- 172.21.15.32:0/2370154367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011830 con 0x7f67ec15ae20 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f1336700 1 -- 172.21.15.32:0/2370154367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8011990 con 0x7f67ec15ae20 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.930+0000 7f67f1336700 1 -- 172.21.15.32:0/2370154367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011d20 con 0x7f67ec15ae20 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 -- 172.21.15.32:0/2370154367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15ae20 msgr2=0x7f67ec0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 --2- 172.21.15.32:0/2370154367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15ae20 0x7f67ec0b29a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009840 tx=0x7f67e8009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 -- 172.21.15.32:0/2370154367 shutdown_connections 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 --2- 172.21.15.32:0/2370154367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15ae20 0x7f67ec0b29a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 -- 172.21.15.32:0/2370154367 >> 172.21.15.32:0/2370154367 conn(0x7f67ec0b57b0 msgr2=0x7f67ec0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 -- 172.21.15.32:0/2370154367 shutdown_connections 2023-11-24T08:00:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.931+0000 7f67f459c700 1 -- 172.21.15.32:0/2370154367 wait complete. 2023-11-24T08:00:15.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.932+0000 7f67f459c700 1 Processor -- start 2023-11-24T08:00:15.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.932+0000 7f67f459c700 1 -- start start 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f459c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67ec0b1a70 0x7f67ec154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f459c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ec15ae20 0x7f67ec154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f459c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 0x7f67ec145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f459c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec15aa50 con 0x7f67ec15b200 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f459c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec1f1ad0 con 0x7f67ec15ae20 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f459c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec151cf0 con 0x7f67ec0b1a70 2023-11-24T08:00:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 0x7f67ec145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 0x7f67ec145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59298/0 (socket says 172.21.15.32:59298) 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 -- 172.21.15.32:0/3426026907 learned_addr learned my addr 172.21.15.32:0/3426026907 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2338700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67ec0b1a70 0x7f67ec154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f1b37700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ec15ae20 0x7f67ec154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 -- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67ec0b1a70 msgr2=0x7f67ec154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67ec0b1a70 0x7f67ec154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 -- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ec15ae20 msgr2=0x7f67ec154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ec15ae20 0x7f67ec154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 -- 172.21.15.32:0/3426026907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e80094a0 con 0x7f67ec15b200 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f1b37700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ec15ae20 0x7f67ec154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2338700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67ec0b1a70 0x7f67ec154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.933+0000 7f67f2b39700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 0x7f67ec145480 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f67e400cb90 tx=0x7f67e400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.934+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4003c00 con 0x7f67ec15b200 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.934+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e4003d60 con 0x7f67ec15b200 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.934+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400abb0 con 0x7f67ec15b200 2023-11-24T08:00:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.934+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec15dbf0 con 0x7f67ec15b200 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.934+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec1433f0 con 0x7f67ec15b200 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.935+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec067f60 con 0x7f67ec15b200 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.935+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67e400ad10 con 0x7f67ec15b200 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.936+0000 7f67deffd700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67d006aed0 0x7f67d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.936+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f67e40893b0 con 0x7f67ec15b200 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.936+0000 7f67f2338700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67d006aed0 0x7f67d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.936+0000 7f67f2338700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67d006aed0 0x7f67d006d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f67e80090b0 tx=0x7f67e8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:15.940+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67e4054760 con 0x7f67ec15b200 2023-11-24T08:00:16.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.140+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f67ec0b39a0 con 0x7f67d006aed0 2023-11-24T08:00:16.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.141+0000 7f67deffd700 1 -- 172.21.15.32:0/3426026907 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f67e8004040 con 0x7f67d006aed0 2023-11-24T08:00:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67d006aed0 msgr2=0x7f67d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67d006aed0 0x7f67d006d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f67e80090b0 tx=0x7f67e8008030 comp rx=0 tx=0).stop 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 msgr2=0x7f67ec145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 0x7f67ec145480 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f67e400cb90 tx=0x7f67e400cf50 comp rx=0 tx=0).stop 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 shutdown_connections 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f67d006aed0 0x7f67d006d390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67ec15b200 0x7f67ec145480 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67ec0b1a70 0x7f67ec154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 --2- 172.21.15.32:0/3426026907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67ec15ae20 0x7f67ec154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 >> 172.21.15.32:0/3426026907 conn(0x7f67ec0b57b0 msgr2=0x7f67ec06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 shutdown_connections 2023-11-24T08:00:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.143+0000 7f67f459c700 1 -- 172.21.15.32:0/3426026907 wait complete. 2023-11-24T08:00:16.147 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T08:00:16.157 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T08:00:16.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.284+0000 7fa673158700 1 Processor -- start 2023-11-24T08:00:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.284+0000 7fa673158700 1 -- start start 2023-11-24T08:00:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa673158700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0cf040 0x7fa66c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa673158700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa66c0d21c0 0x7fa66c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa673158700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c0d01c0 con 0x7fa66c0702e0 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa673158700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c070910 con 0x7fa66c0d21c0 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa673158700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c070bf0 con 0x7fa66c0cf040 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa6716f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa66c0d21c0 0x7fa66c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa66bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa66c0702e0 0x7fa66c0d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32782/0 (socket says 172.21.15.32:32782) 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.285+0000 7fa66bfff700 1 -- 172.21.15.32:0/688556594 learned_addr learned my addr 172.21.15.32:0/688556594 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa670ef4700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0cf040 0x7fa66c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa6716f5700 1 -- 172.21.15.32:0/688556594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0cf040 msgr2=0x7fa66c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa6716f5700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0cf040 0x7fa66c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa6716f5700 1 -- 172.21.15.32:0/688556594 >> v1:172.21.15.32:6789/0 conn(0x7fa66c0702e0 legacy=0x7fa66c0d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa6716f5700 1 -- 172.21.15.32:0/688556594 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0aebb0 con 0x7fa66c0d21c0 2023-11-24T08:00:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa670ef4700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0cf040 0x7fa66c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa66b7fe700 1 -- 172.21.15.32:0/688556594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113198848 0 0) 0x7fa66c0d01c0 con 0x7fa66c0702e0 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa6716f5700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c0af3b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa664009d90 tx=0x7fa66400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f41a753c19acc68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa66b7fe700 1 -- 172.21.15.32:0/688556594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66400e070 con 0x7fa66c0d21c0 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa66b7fe700 1 -- 172.21.15.32:0/688556594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66400bbd0 con 0x7fa66c0d21c0 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.286+0000 7fa66b7fe700 1 -- 172.21.15.32:0/688556594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664012430 con 0x7fa66c0d21c0 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 -- 172.21.15.32:0/688556594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 msgr2=0x7fa66c0af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c0af3b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa664009d90 tx=0x7fa66400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 -- 172.21.15.32:0/688556594 shutdown_connections 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c0af3b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 --2- 172.21.15.32:0/688556594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0cf040 0x7fa66c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 -- 172.21.15.32:0/688556594 >> 172.21.15.32:0/688556594 conn(0x7fa66c09aa00 msgr2=0x7fa66c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 -- 172.21.15.32:0/688556594 shutdown_connections 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.287+0000 7fa673158700 1 -- 172.21.15.32:0/688556594 wait complete. 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.288+0000 7fa673158700 1 Processor -- start 2023-11-24T08:00:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.288+0000 7fa673158700 1 -- start start 2023-11-24T08:00:16.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa673158700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0702e0 0x7fa66c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa673158700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c0cf040 0x7fa66c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa673158700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa673158700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0c6430 con 0x7fa66c0d21c0 2023-11-24T08:00:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa673158700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0c57b0 con 0x7fa66c0cf040 2023-11-24T08:00:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa673158700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0c5a90 con 0x7fa66c0702e0 2023-11-24T08:00:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c15ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59312/0 (socket says 172.21.15.32:59312) 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 -- 172.21.15.32:0/3703990216 learned_addr learned my addr 172.21.15.32:0/3703990216 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa66bfff700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c0cf040 0x7fa66c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa670ef4700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0702e0 0x7fa66c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 -- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0702e0 msgr2=0x7fa66c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa66c0702e0 0x7fa66c0c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 -- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c0cf040 msgr2=0x7fa66c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c0cf040 0x7fa66c15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 -- 172.21.15.32:0/3703990216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa664009a70 con 0x7fa66c0d21c0 2023-11-24T08:00:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6716f5700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c15ca80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06e5d0 tx=0x7fa664004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.289+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664012430 con 0x7fa66c0d21c0 2023-11-24T08:00:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.290+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664020070 con 0x7fa66c0d21c0 2023-11-24T08:00:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.290+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c0c5d70 con 0x7fa66c0d21c0 2023-11-24T08:00:16.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.290+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664004280 con 0x7fa66c0d21c0 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.290+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c15c2c0 con 0x7fa66c0d21c0 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.290+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa66c15f000 con 0x7fa66c0d21c0 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.291+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6640043e0 con 0x7fa66c0d21c0 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.291+0000 7fa6697fa700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa65006aed0 0x7fa65006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.291+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa664022070 con 0x7fa66c0d21c0 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa6697fa700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7fa650070d30 0x7fa650073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa670ef4700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa65006aed0 0x7fa65006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 --> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa650073820 con 0x7fa650070d30 2023-11-24T08:00:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa66408d960 con 0x7fa66c0d21c0 2023-11-24T08:00:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa66bfff700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7fa650070d30 0x7fa650073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa670ef4700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa65006aed0 0x7fa65006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0b3ee0 tx=0x7fa660008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.292+0000 7fa66bfff700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7fa650070d30 0x7fa650073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.293+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== osd.6 v2:172.21.15.123:6800/2373254053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa650073820 con 0x7fa650070d30 2023-11-24T08:00:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.308+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 --> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa66c0cf420 con 0x7fa650070d30 2023-11-24T08:00:16.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa6697fa700 1 -- 172.21.15.32:0/3703990216 <== osd.6 v2:172.21.15.123:6800/2373254053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa658004b10 con 0x7fa650070d30 2023-11-24T08:00:16.311 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:16.311 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7fa650070d30 msgr2=0x7fa650073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6800/2373254053,v1:172.21.15.123:6801/2373254053] conn(0x7fa650070d30 0x7fa650073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa65006aed0 msgr2=0x7fa65006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa65006aed0 0x7fa65006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0b3ee0 tx=0x7fa660008040 comp rx=0 tx=0).stop 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 msgr2=0x7fa66c15ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 --2- 172.21.15.32:0/3703990216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0d21c0 0x7fa66c15ca80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06e5d0 tx=0x7fa664004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa6716f5700 1 -- 172.21.15.32:0/3703990216 reap_dead start 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 shutdown_connections 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 >> 172.21.15.32:0/3703990216 conn(0x7fa66c09aa00 msgr2=0x7fa66c06fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 shutdown_connections 2023-11-24T08:00:16.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.309+0000 7fa673158700 1 -- 172.21.15.32:0/3703990216 wait complete. 2023-11-24T08:00:16.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T08:00:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.447+0000 7f6032151700 1 Processor -- start 2023-11-24T08:00:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.447+0000 7f6032151700 1 -- start start 2023-11-24T08:00:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f6032151700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c15ddd0 0x7f602c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f6032151700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f602c15e1b0 0x7f602c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f6032151700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c0b1d40 con 0x7f602c0d1b30 2023-11-24T08:00:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f6032151700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602c0b28c0 con 0x7f602c15e1b0 2023-11-24T08:00:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f6032151700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c06c000 con 0x7f602c15ddd0 2023-11-24T08:00:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f602c15e1b0 0x7f602c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f602c15e1b0 0x7f602c0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59318/0 (socket says 172.21.15.32:59318) 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 -- 172.21.15.32:0/2137558589 learned_addr learned my addr 172.21.15.32:0/2137558589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602b7fe700 1 --2- 172.21.15.32:0/2137558589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c15ddd0 0x7f602c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 -- 172.21.15.32:0/2137558589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c15ddd0 msgr2=0x7f602c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 --2- 172.21.15.32:0/2137558589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c15ddd0 0x7f602c0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 -- 172.21.15.32:0/2137558589 >> v1:172.21.15.123:6789/0 conn(0x7f602c0d1b30 legacy=0x7f602c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 -- 172.21.15.32:0/2137558589 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c0b1f90 con 0x7f602c15e1b0 2023-11-24T08:00:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.448+0000 7f602affd700 1 --2- 172.21.15.32:0/2137558589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15e1b0 0x7f602c0cf000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f601c0098f0 tx=0x7f601c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86087a0315226ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f602a7fc700 1 -- 172.21.15.32:0/2137558589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c011830 con 0x7f602c15e1b0 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f602a7fc700 1 -- 172.21.15.32:0/2137558589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c011990 con 0x7f602c15e1b0 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f602a7fc700 1 -- 172.21.15.32:0/2137558589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c011d20 con 0x7f602c15e1b0 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 -- 172.21.15.32:0/2137558589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15e1b0 msgr2=0x7f602c0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 --2- 172.21.15.32:0/2137558589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15e1b0 0x7f602c0cf000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f601c0098f0 tx=0x7f601c009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 -- 172.21.15.32:0/2137558589 shutdown_connections 2023-11-24T08:00:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 --2- 172.21.15.32:0/2137558589 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c15ddd0 0x7f602c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 --2- 172.21.15.32:0/2137558589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15e1b0 0x7f602c0cf000 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 -- 172.21.15.32:0/2137558589 >> 172.21.15.32:0/2137558589 conn(0x7f602c0731a0 msgr2=0x7f602c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 -- 172.21.15.32:0/2137558589 shutdown_connections 2023-11-24T08:00:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.449+0000 7f6032151700 1 -- 172.21.15.32:0/2137558589 wait complete. 2023-11-24T08:00:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.451+0000 7f6032151700 1 Processor -- start 2023-11-24T08:00:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.451+0000 7f6032151700 1 -- start start 2023-11-24T08:00:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f6032151700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c0d1b30 0x7f602c155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f6032151700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 0x7f602c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f6032151700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f602c15e1b0 0x7f602c09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f6032151700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c155680 con 0x7f602c15ddd0 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f6032151700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0c60a0 con 0x7f602c15e1b0 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f6032151700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0c6380 con 0x7f602c0d1b30 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 0x7f602c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 0x7f602c0c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59326/0 (socket says 172.21.15.32:59326) 2023-11-24T08:00:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 -- 172.21.15.32:0/2301202745 learned_addr learned my addr 172.21.15.32:0/2301202745 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602b7fe700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c0d1b30 0x7f602c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602bfff700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f602c15e1b0 0x7f602c09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 -- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c0d1b30 msgr2=0x7f602c155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c0d1b30 0x7f602c155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 -- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f602c15e1b0 msgr2=0x7f602c09d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f602c15e1b0 0x7f602c09d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.452+0000 7f602affd700 1 -- 172.21.15.32:0/2301202745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009550 con 0x7f602c15ddd0 2023-11-24T08:00:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.453+0000 7f602affd700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 0x7f602c0c7f50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c00 tx=0x7f601c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.453+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c004530 con 0x7f602c15ddd0 2023-11-24T08:00:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.453+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c01f070 con 0x7f602c15ddd0 2023-11-24T08:00:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.453+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c0b2ed0 con 0x7f602c15ddd0 2023-11-24T08:00:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.453+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0038c0 con 0x7f602c15ddd0 2023-11-24T08:00:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.453+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c0c6940 con 0x7f602c15ddd0 2023-11-24T08:00:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.454+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c067f60 con 0x7f602c15ddd0 2023-11-24T08:00:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.454+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f601c021030 con 0x7f602c15ddd0 2023-11-24T08:00:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.455+0000 7f6023fff700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f600c06aed0 0x7f600c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.455+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f601c08a790 con 0x7f602c15ddd0 2023-11-24T08:00:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.455+0000 7f602b7fe700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f600c06aed0 0x7f600c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.455+0000 7f602b7fe700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f600c06aed0 0x7f600c06d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6014009ab0 tx=0x7f6014008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.459+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f601c055c50 con 0x7f602c15ddd0 2023-11-24T08:00:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.664+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f602c0b2d50 con 0x7f600c06aed0 2023-11-24T08:00:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.665+0000 7f6023fff700 1 -- 172.21.15.32:0/2301202745 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f60140106e0 con 0x7f600c06aed0 2023-11-24T08:00:16.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f600c06aed0 msgr2=0x7f600c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f600c06aed0 0x7f600c06d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6014009ab0 tx=0x7f6014008040 comp rx=0 tx=0).stop 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 msgr2=0x7f602c0c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 0x7f602c0c7f50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c00 tx=0x7f601c004750 comp rx=0 tx=0).stop 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 shutdown_connections 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f600c06aed0 0x7f600c06d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f602c15e1b0 0x7f602c09d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f602c0d1b30 0x7f602c155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 --2- 172.21.15.32:0/2301202745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f602c15ddd0 0x7f602c0c7f50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 >> 172.21.15.32:0/2301202745 conn(0x7f602c0731a0 msgr2=0x7f602c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 shutdown_connections 2023-11-24T08:00:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.667+0000 7f6032151700 1 -- 172.21.15.32:0/2301202745 wait complete. 2023-11-24T08:00:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T08:00:16.681 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T08:00:16.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:16.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 Processor -- start 2023-11-24T08:00:16.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 -- start start 2023-11-24T08:00:16.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 0x7f741c15c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 0x7f741c0ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c06dea0 con 0x7f741c06f2e0 2023-11-24T08:00:16.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c06e140 con 0x7f741c155780 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.809+0000 7f742262e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c06dbd0 con 0x7f741c06eee0 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 0x7f741c15c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 0x7f741c0ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 0x7f741c15c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.32:43482/0 (socket says 172.21.15.32:43482) 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741bfff700 1 -- 172.21.15.32:0/3671084429 learned_addr learned my addr 172.21.15.32:0/3671084429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 0x7f741c0ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.32:54586/0 (socket says 172.21.15.32:54586) 2023-11-24T08:00:16.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f7420bcb700 1 --1- 172.21.15.32:0/3671084429 >> v1:172.21.15.123:6789/0 conn(0x7f741c06f2e0 0x7f741c15c9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.32:47802/0 (socket says 172.21.15.32:47802) 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 -- 172.21.15.32:0/3671084429 >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 msgr2=0x7f741c15c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 --2- 172.21.15.32:0/3671084429 >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 0x7f741c15c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 -- 172.21.15.32:0/3671084429 >> v1:172.21.15.123:6789/0 conn(0x7f741c06f2e0 legacy=0x7f741c15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 -- 172.21.15.32:0/3671084429 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c06e400 con 0x7f741c06eee0 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741bfff700 1 --2- 172.21.15.32:0/3671084429 >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 0x7f741c15c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.810+0000 7f741b7fe700 1 --2- 172.21.15.32:0/3671084429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 0x7f741c0ce9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7410009840 tx=0x7f7410009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f4d1b87e187145 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f741affd700 1 -- 172.21.15.32:0/3671084429 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410011830 con 0x7f741c06eee0 2023-11-24T08:00:16.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f741affd700 1 -- 172.21.15.32:0/3671084429 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7410011990 con 0x7f741c06eee0 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f741affd700 1 -- 172.21.15.32:0/3671084429 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410011d20 con 0x7f741c06eee0 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 -- 172.21.15.32:0/3671084429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 msgr2=0x7f741c0ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 --2- 172.21.15.32:0/3671084429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 0x7f741c0ce9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7410009840 tx=0x7f7410009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 -- 172.21.15.32:0/3671084429 shutdown_connections 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 --2- 172.21.15.32:0/3671084429 >> v2:172.21.15.123:3300/0 conn(0x7f741c155780 0x7f741c15c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 --2- 172.21.15.32:0/3671084429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06eee0 0x7f741c0ce9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 -- 172.21.15.32:0/3671084429 >> 172.21.15.32:0/3671084429 conn(0x7f741c0d0af0 msgr2=0x7f741c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 -- 172.21.15.32:0/3671084429 shutdown_connections 2023-11-24T08:00:16.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.811+0000 7f742262e700 1 -- 172.21.15.32:0/3671084429 wait complete. 2023-11-24T08:00:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 Processor -- start 2023-11-24T08:00:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 -- start start 2023-11-24T08:00:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741c06eee0 0x7f741c151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06f2e0 0x7f741c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f741c155780 0x7f741c0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0b48b0 con 0x7f741c06eee0 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0b4a90 con 0x7f741c155780 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.813+0000 7f742262e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0b4d70 con 0x7f741c06f2e0 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741c06eee0 0x7f741c151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741c06eee0 0x7f741c151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59338/0 (socket says 172.21.15.32:59338) 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 -- 172.21.15.32:0/576607819 learned_addr learned my addr 172.21.15.32:0/576607819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f7420bcb700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f741c155780 0x7f741c0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741b7fe700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06f2e0 0x7f741c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 -- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06f2e0 msgr2=0x7f741c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f741c06f2e0 0x7f741c0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 -- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f741c155780 msgr2=0x7f741c0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f741c155780 0x7f741c0b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 -- 172.21.15.32:0/576607819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c009550 con 0x7f741c06eee0 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f741bfff700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741c06eee0 0x7f741c151670 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f740c00e9b0 tx=0x7f740c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00c850 con 0x7f741c06eee0 2023-11-24T08:00:16.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00c9b0 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74100094a0 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00cd20 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.814+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c0b5330 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.815+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f741c15c750 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f740c013470 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f7418ff9700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f740806aed0 0x7f740806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f740c0897c0 con 0x7f741c06eee0 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f7418ff9700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7f7408070d30 0x7f7408073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 --> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7408073820 con 0x7f7408070d30 2023-11-24T08:00:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f740c089c10 con 0x7f741c06eee0 2023-11-24T08:00:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.816+0000 7f741b7fe700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f740806aed0 0x7f740806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.817+0000 7f7420bcb700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7f7408070d30 0x7f7408073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.817+0000 7f741b7fe700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f740806aed0 0x7f740806d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7410000c00 tx=0x7f7410008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.817+0000 7f7420bcb700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7f7408070d30 0x7f7408073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.818+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== osd.7 v2:172.21.15.123:6816/3074243597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7408073820 con 0x7f7408070d30 2023-11-24T08:00:16.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.834+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 --> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f741c06e9c0 con 0x7f7408070d30 2023-11-24T08:00:16.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.834+0000 7f7418ff9700 1 -- 172.21.15.32:0/576607819 <== osd.7 v2:172.21.15.123:6816/3074243597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7414007130 con 0x7f7408070d30 2023-11-24T08:00:16.836 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-11-24T08:00:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:16.836 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-11-24T08:00:16.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7f7408070d30 msgr2=0x7f7408073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6816/3074243597,v1:172.21.15.123:6817/3074243597] conn(0x7f7408070d30 0x7f7408073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f740806aed0 msgr2=0x7f740806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f740806aed0 0x7f740806d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7410000c00 tx=0x7f7410008030 comp rx=0 tx=0).stop 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741c06eee0 msgr2=0x7f741c151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 --2- 172.21.15.32:0/576607819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f741c06eee0 0x7f741c151670 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f740c00e9b0 tx=0x7f740c00ed70 comp rx=0 tx=0).stop 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f7420bcb700 1 -- 172.21.15.32:0/576607819 reap_dead start 2023-11-24T08:00:16.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 shutdown_connections 2023-11-24T08:00:16.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 >> 172.21.15.32:0/576607819 conn(0x7f741c0d0af0 msgr2=0x7f741c15c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 shutdown_connections 2023-11-24T08:00:16.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.835+0000 7f742262e700 1 -- 172.21.15.32:0/576607819 wait complete. 2023-11-24T08:00:16.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T08:00:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.972+0000 7f3bcb876700 1 Processor -- start 2023-11-24T08:00:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 -- start start 2023-11-24T08:00:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc41446a0 0x7f3bc40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3bc40adf00 0x7f3bc406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bc406c540 0x7f3bc406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc4144a80 con 0x7f3bc406c540 2023-11-24T08:00:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc406c9b0 con 0x7f3bc40adf00 2023-11-24T08:00:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bcb876700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40af950 con 0x7f3bc41446a0 2023-11-24T08:00:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.973+0000 7f3bc9e13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bc406c540 0x7f3bc406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bc406c540 0x7f3bc406da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59352/0 (socket says 172.21.15.32:59352) 2023-11-24T08:00:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 -- 172.21.15.32:0/422257558 learned_addr learned my addr 172.21.15.32:0/422257558 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc8e11700 1 --2- 172.21.15.32:0/422257558 >> v2:172.21.15.123:3300/0 conn(0x7f3bc40adf00 0x7f3bc406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9612700 1 --2- 172.21.15.32:0/422257558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc41446a0 0x7f3bc40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 -- 172.21.15.32:0/422257558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc41446a0 msgr2=0x7f3bc40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 --2- 172.21.15.32:0/422257558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc41446a0 0x7f3bc40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 -- 172.21.15.32:0/422257558 >> v2:172.21.15.123:3300/0 conn(0x7f3bc40adf00 msgr2=0x7f3bc406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 --2- 172.21.15.32:0/422257558 >> v2:172.21.15.123:3300/0 conn(0x7f3bc40adf00 0x7f3bc406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 -- 172.21.15.32:0/422257558 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40afc30 con 0x7f3bc406c540 2023-11-24T08:00:16.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bc9e13700 1 --2- 172.21.15.32:0/422257558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc406c540 0x7f3bc406da30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3bc406d470 tx=0x7f3bbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4c993c6091b373 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bbbfff700 1 -- 172.21.15.32:0/422257558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00e070 con 0x7f3bc406c540 2023-11-24T08:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bbbfff700 1 -- 172.21.15.32:0/422257558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00b980 con 0x7f3bc406c540 2023-11-24T08:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.974+0000 7f3bbbfff700 1 -- 172.21.15.32:0/422257558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00bd30 con 0x7f3bc406c540 2023-11-24T08:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 -- 172.21.15.32:0/422257558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc406c540 msgr2=0x7f3bc406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 --2- 172.21.15.32:0/422257558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc406c540 0x7f3bc406da30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3bc406d470 tx=0x7f3bbc00b330 comp rx=0 tx=0).stop 2023-11-24T08:00:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 -- 172.21.15.32:0/422257558 shutdown_connections 2023-11-24T08:00:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 --2- 172.21.15.32:0/422257558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc406c540 0x7f3bc406da30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 --2- 172.21.15.32:0/422257558 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc41446a0 0x7f3bc40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 --2- 172.21.15.32:0/422257558 >> v2:172.21.15.123:3300/0 conn(0x7f3bc40adf00 0x7f3bc406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 -- 172.21.15.32:0/422257558 >> 172.21.15.32:0/422257558 conn(0x7f3bc4071b90 msgr2=0x7f3bc4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 -- 172.21.15.32:0/422257558 shutdown_connections 2023-11-24T08:00:16.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.975+0000 7f3bcb876700 1 -- 172.21.15.32:0/422257558 wait complete. 2023-11-24T08:00:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.976+0000 7f3bcb876700 1 Processor -- start 2023-11-24T08:00:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.976+0000 7f3bcb876700 1 -- start start 2023-11-24T08:00:16.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bcb876700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bc406c540 0x7f3bc41575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bcb876700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc40adf00 0x7f3bc415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bcb876700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 0x7f3bc415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bcb876700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4143760 con 0x7f3bc41446a0 2023-11-24T08:00:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bcb876700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4143480 con 0x7f3bc406c540 2023-11-24T08:00:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bcb876700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc413e840 con 0x7f3bc40adf00 2023-11-24T08:00:16.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 0x7f3bc415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 0x7f3bc415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59358/0 (socket says 172.21.15.32:59358) 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 -- 172.21.15.32:0/3448482316 learned_addr learned my addr 172.21.15.32:0/3448482316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9612700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bc406c540 0x7f3bc41575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc8e11700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc40adf00 0x7f3bc415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 -- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc40adf00 msgr2=0x7f3bc415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc40adf00 0x7f3bc415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 -- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bc406c540 msgr2=0x7f3bc41575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bc406c540 0x7f3bc41575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 -- 172.21.15.32:0/3448482316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009b30 con 0x7f3bc41446a0 2023-11-24T08:00:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bc9e13700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 0x7f3bc415de60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40729d0 tx=0x7f3bbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.977+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004250 con 0x7f3bc41446a0 2023-11-24T08:00:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.978+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc0043b0 con 0x7f3bc41446a0 2023-11-24T08:00:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.978+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc413eb20 con 0x7f3bc41446a0 2023-11-24T08:00:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.978+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004760 con 0x7f3bc41446a0 2023-11-24T08:00:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.978+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc4157af0 con 0x7f3bc41446a0 2023-11-24T08:00:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.979+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc413f170 con 0x7f3bc41446a0 2023-11-24T08:00:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.979+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3bbc01c440 con 0x7f3bc41446a0 2023-11-24T08:00:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.980+0000 7f3bb9ffb700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3ba806aed0 0x7f3ba806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.980+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3bbc08a7d0 con 0x7f3bc41446a0 2023-11-24T08:00:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.980+0000 7f3bc9612700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3ba806aed0 0x7f3ba806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.980+0000 7f3bc9612700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3ba806aed0 0x7f3ba806d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4068780 tx=0x7f3bb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:16.984+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bbc055cc0 con 0x7f3bc41446a0 2023-11-24T08:00:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.184+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc40b04d0 con 0x7f3ba806aed0 2023-11-24T08:00:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.185+0000 7f3bb9ffb700 1 -- 172.21.15.32:0/3448482316 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3bb400b5c0 con 0x7f3ba806aed0 2023-11-24T08:00:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3ba806aed0 msgr2=0x7f3ba806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3ba806aed0 0x7f3ba806d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4068780 tx=0x7f3bb4008040 comp rx=0 tx=0).stop 2023-11-24T08:00:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 msgr2=0x7f3bc415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 0x7f3bc415de60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40729d0 tx=0x7f3bbc004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 shutdown_connections 2023-11-24T08:00:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f3ba806aed0 0x7f3ba806d390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc41446a0 0x7f3bc415de60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bc406c540 0x7f3bc41575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 --2- 172.21.15.32:0/3448482316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3bc40adf00 0x7f3bc415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 >> 172.21.15.32:0/3448482316 conn(0x7f3bc4071b90 msgr2=0x7f3bc406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 shutdown_connections 2023-11-24T08:00:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.187+0000 7f3bcb876700 1 -- 172.21.15.32:0/3448482316 wait complete. 2023-11-24T08:00:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T08:00:17.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:17.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.324+0000 7f8e73c8e700 1 Processor -- start 2023-11-24T08:00:17.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.324+0000 7f8e73c8e700 1 -- start start 2023-11-24T08:00:17.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e73c8e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8e6c0b0ba0 0x7f8e6c0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e73c8e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e6c15ee90 0x7f8e6c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e73c8e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0b2bd0 con 0x7f8e6c0b0f80 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e73c8e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c0b1fc0 con 0x7f8e6c15ee90 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e73c8e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c0b22a0 con 0x7f8e6c0b0ba0 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e7222b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e6c15ee90 0x7f8e6c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e7222b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e6c15ee90 0x7f8e6c15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59366/0 (socket says 172.21.15.32:59366) 2023-11-24T08:00:17.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e7222b700 1 -- 172.21.15.32:0/3407331856 learned_addr learned my addr 172.21.15.32:0/3407331856 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.325+0000 7f8e71a2a700 1 --2- 172.21.15.32:0/3407331856 >> v2:172.21.15.123:3300/0 conn(0x7f8e6c0b0ba0 0x7f8e6c0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e7222b700 1 -- 172.21.15.32:0/3407331856 >> v2:172.21.15.123:3300/0 conn(0x7f8e6c0b0ba0 msgr2=0x7f8e6c0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e7222b700 1 --2- 172.21.15.32:0/3407331856 >> v2:172.21.15.123:3300/0 conn(0x7f8e6c0b0ba0 0x7f8e6c0af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e7222b700 1 -- 172.21.15.32:0/3407331856 >> v1:172.21.15.123:6789/0 conn(0x7f8e6c0b0f80 legacy=0x7f8e6c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e7222b700 1 -- 172.21.15.32:0/3407331856 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c0b2580 con 0x7f8e6c15ee90 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e7222b700 1 --2- 172.21.15.32:0/3407331856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c15b450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0d1340 tx=0x7f8e6400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1867c2cfa6af1ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e70a28700 1 -- 172.21.15.32:0/3407331856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64010070 con 0x7f8e6c15ee90 2023-11-24T08:00:17.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e70a28700 1 -- 172.21.15.32:0/3407331856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6400dce0 con 0x7f8e6c15ee90 2023-11-24T08:00:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e70a28700 1 -- 172.21.15.32:0/3407331856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400b310 con 0x7f8e6c15ee90 2023-11-24T08:00:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e73c8e700 1 -- 172.21.15.32:0/3407331856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 msgr2=0x7f8e6c15b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.326+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/3407331856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c15b450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0d1340 tx=0x7f8e6400d430 comp rx=0 tx=0).stop 2023-11-24T08:00:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.327+0000 7f8e73c8e700 1 -- 172.21.15.32:0/3407331856 shutdown_connections 2023-11-24T08:00:17.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.327+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/3407331856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c15b450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.327+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/3407331856 >> v2:172.21.15.123:3300/0 conn(0x7f8e6c0b0ba0 0x7f8e6c0af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.327+0000 7f8e73c8e700 1 -- 172.21.15.32:0/3407331856 >> 172.21.15.32:0/3407331856 conn(0x7f8e6c0b57b0 msgr2=0x7f8e6c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.327+0000 7f8e73c8e700 1 -- 172.21.15.32:0/3407331856 shutdown_connections 2023-11-24T08:00:17.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.327+0000 7f8e73c8e700 1 -- 172.21.15.32:0/3407331856 wait complete. 2023-11-24T08:00:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.328+0000 7f8e73c8e700 1 Processor -- start 2023-11-24T08:00:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.328+0000 7f8e73c8e700 1 -- start start 2023-11-24T08:00:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e6c0b0ba0 0x7f8e6c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e6c0b0f80 0x7f8e6c0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0cb9d0 con 0x7f8e6c15ee90 2023-11-24T08:00:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0cadc0 con 0x7f8e6c0b0f80 2023-11-24T08:00:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0cb0a0 con 0x7f8e6c0b0ba0 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59372/0 (socket says 172.21.15.32:59372) 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 -- 172.21.15.32:0/788276651 learned_addr learned my addr 172.21.15.32:0/788276651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e71229700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e6c0b0f80 0x7f8e6c0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e71a2a700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e6c0b0ba0 0x7f8e6c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 -- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e6c0b0ba0 msgr2=0x7f8e6c0b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e6c0b0ba0 0x7f8e6c0b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 -- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e6c0b0f80 msgr2=0x7f8e6c0c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e6c0b0f80 0x7f8e6c0c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 -- 172.21.15.32:0/788276651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6400bd40 con 0x7f8e6c15ee90 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e71a2a700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e6c0b0ba0 0x7f8e6c0b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e7222b700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c1515d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0722a0 tx=0x7f8e64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64009a10 con 0x7f8e6c15ee90 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e64009b70 con 0x7f8e6c15ee90 2023-11-24T08:00:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c0cb380 con 0x7f8e6c15ee90 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400b4c0 con 0x7f8e6c15ee90 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.329+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c0b5090 con 0x7f8e6c15ee90 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.330+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c067f60 con 0x7f8e6c15ee90 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.331+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e64004040 con 0x7f8e6c15ee90 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.331+0000 7f8e627fc700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8e5006aed0 0x7f8e5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.331+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8e6408a7d0 con 0x7f8e6c15ee90 2023-11-24T08:00:17.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.331+0000 7f8e71a2a700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8e5006aed0 0x7f8e5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.332+0000 7f8e71a2a700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8e5006aed0 0x7f8e5006d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0cf2f0 tx=0x7f8e5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.336+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e64055c10 con 0x7f8e6c15ee90 2023-11-24T08:00:17.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.531+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e6c15ebd0 con 0x7f8e5006aed0 2023-11-24T08:00:17.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.533+0000 7f8e627fc700 1 -- 172.21.15.32:0/788276651 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50004 (secure 0 0 0) 0x7f8e6c15ebd0 con 0x7f8e5006aed0 2023-11-24T08:00:17.536 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:17.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.535+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8e5006aed0 msgr2=0x7f8e5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8e5006aed0 0x7f8e5006d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0cf2f0 tx=0x7f8e5c008040 comp rx=0 tx=0).stop 2023-11-24T08:00:17.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 msgr2=0x7f8e6c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c1515d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0722a0 tx=0x7f8e64004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 shutdown_connections 2023-11-24T08:00:17.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f8e5006aed0 0x7f8e5006d390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e6c15ee90 0x7f8e6c1515d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e6c0b0ba0 0x7f8e6c0b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 --2- 172.21.15.32:0/788276651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e6c0b0f80 0x7f8e6c0c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 >> 172.21.15.32:0/788276651 conn(0x7f8e6c0b57b0 msgr2=0x7f8e6c15aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 shutdown_connections 2023-11-24T08:00:17.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:17.536+0000 7f8e73c8e700 1 -- 172.21.15.32:0/788276651 wait complete. 2023-11-24T08:00:17.540 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T08:00:17.549 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-11-24T08:00:16.554530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220520,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754200,"statfs":{"total":773094113280,"available":772868300800,"internally_reserved":0,"allocated":4431872,"data_stored":2432112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003246"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T07:59:54.780239+0000","last_change":"2023-11-24T07:58:34.470719+0000","last_active":"2023-11-24T07:59:54.780239+0000","last_peered":"2023-11-24T07:59:54.780239+0000","last_clean":"2023-11-24T07:59:54.780239+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T07:59:54.780239+0000","last_undegraded":"2023-11-24T07:59:54.780239+0000","last_fullsized":"2023-11-24T07:59:54.780239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:32.986828+0000","last_clean_scrub_stamp":"2023-11-24T07:58:32.986828+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:05:46.652566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.80500000000000005}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.77500000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.62}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.38200000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.434},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.626},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.307},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.502},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.433},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.45100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.433}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.377}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.622},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.67000000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.375},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.434}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.628}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.498}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.434}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.317},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34499999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.379}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.36399999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:17.554 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T07:58:32.986828+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=7, tm_min=58, tm_sec=32, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=8, tm_min=0, tm_sec=12, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T08:00:17.554 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T08:00:37.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:37.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.693+0000 7fe7b5b26700 1 Processor -- start 2023-11-24T08:00:37.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.693+0000 7fe7b5b26700 1 -- start start 2023-11-24T08:00:37.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7b5b26700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7b006df20 0x7fe7b006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:37.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7b5b26700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b006ee70 con 0x7fe7b015ffb0 2023-11-24T08:00:37.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7b5b26700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b006d4d0 con 0x7fe7b0145760 2023-11-24T08:00:37.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7b5b26700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b00cc5e0 con 0x7fe7b006df20 2023-11-24T08:00:37.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7aeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7b006df20 0x7fe7b006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:37.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7affff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7b015ffb0 0x7fe7b00cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32860/0 (socket says 172.21.15.32:32860) 2023-11-24T08:00:37.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7affff700 1 -- 172.21.15.32:0/4220386767 learned_addr learned my addr 172.21.15.32:0/4220386767 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:37.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7aeffd700 1 -- 172.21.15.32:0/4220386767 >> v1:172.21.15.123:6789/0 conn(0x7fe7b0145760 legacy=0x7fe7b015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:37.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7aeffd700 1 -- 172.21.15.32:0/4220386767 >> v1:172.21.15.32:6789/0 conn(0x7fe7b015ffb0 legacy=0x7fe7b00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7ae7fc700 1 -- 172.21.15.32:0/4220386767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789994120 0 0) 0x7fe7b006ee70 con 0x7fe7b015ffb0 2023-11-24T08:00:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.694+0000 7fe7aeffd700 1 -- 172.21.15.32:0/4220386767 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00cc8c0 con 0x7fe7b006df20 2023-11-24T08:00:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7aeffd700 1 --2- 172.21.15.32:0/4220386767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b006e930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe798009840 tx=0x7fe798009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cc0673e2479cd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7ae7fc700 1 -- 172.21.15.32:0/4220386767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798004230 con 0x7fe7b006df20 2023-11-24T08:00:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7ae7fc700 1 -- 172.21.15.32:0/4220386767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe798004390 con 0x7fe7b006df20 2023-11-24T08:00:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7ae7fc700 1 -- 172.21.15.32:0/4220386767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980036a0 con 0x7fe7b006df20 2023-11-24T08:00:37.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 -- 172.21.15.32:0/4220386767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 msgr2=0x7fe7b006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:37.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/4220386767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b006e930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe798009840 tx=0x7fe798009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:37.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 -- 172.21.15.32:0/4220386767 shutdown_connections 2023-11-24T08:00:37.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/4220386767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b006e930 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 -- 172.21.15.32:0/4220386767 >> 172.21.15.32:0/4220386767 conn(0x7fe7b00b1740 msgr2=0x7fe7b00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 -- 172.21.15.32:0/4220386767 shutdown_connections 2023-11-24T08:00:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.695+0000 7fe7b5b26700 1 -- 172.21.15.32:0/4220386767 wait complete. 2023-11-24T08:00:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.697+0000 7fe7b5b26700 1 Processor -- start 2023-11-24T08:00:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.697+0000 7fe7b5b26700 1 -- start start 2023-11-24T08:00:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7b5b26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7b5b26700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7b0145760 0x7fe7b014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7b5b26700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7b015ffb0 0x7fe7b014d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7b5b26700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b014bcd0 con 0x7fe7b006df20 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7b5b26700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b014bfb0 con 0x7fe7b0145760 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7b5b26700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b014c290 con 0x7fe7b015ffb0 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59382/0 (socket says 172.21.15.32:59382) 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 -- 172.21.15.32:0/1764903110 learned_addr learned my addr 172.21.15.32:0/1764903110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7aeffd700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7b0145760 0x7fe7b014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7affff700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7b015ffb0 0x7fe7b014d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 -- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7b015ffb0 msgr2=0x7fe7b014d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7b015ffb0 0x7fe7b014d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 -- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7b0145760 msgr2=0x7fe7b014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7b0145760 0x7fe7b014cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.698+0000 7fe7af7fe700 1 -- 172.21.15.32:0/1764903110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7980094a0 con 0x7fe7b006df20 2023-11-24T08:00:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.699+0000 7fe7af7fe700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b015ec70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c4d0 tx=0x7fe7a000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.699+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0011790 con 0x7fe7b006df20 2023-11-24T08:00:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.699+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a00118f0 con 0x7fe7b006df20 2023-11-24T08:00:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.699+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b014c570 con 0x7fe7b006df20 2023-11-24T08:00:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.699+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0011ca0 con 0x7fe7b006df20 2023-11-24T08:00:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.699+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b015f1b0 con 0x7fe7b006df20 2023-11-24T08:00:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.700+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b014ca90 con 0x7fe7b006df20 2023-11-24T08:00:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.700+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7a001c460 con 0x7fe7b006df20 2023-11-24T08:00:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.701+0000 7fe7a7fff700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe79006aed0 0x7fe79006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.701+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe7a0089150 con 0x7fe7b006df20 2023-11-24T08:00:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.701+0000 7fe7aeffd700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe79006aed0 0x7fe79006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.702+0000 7fe7aeffd700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe79006aed0 0x7fe79006d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe798009840 tx=0x7fe79801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:37.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.706+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe7b014ca90 con 0x7fe7b006df20 2023-11-24T08:00:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.904+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 --> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7b014c120 con 0x7fe79006aed0 2023-11-24T08:00:37.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.907+0000 7fe7a7fff700 1 -- 172.21.15.32:0/1764903110 <== mgr.4105 v2:172.21.15.123:6832/1302929475 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50128 (secure 0 0 0) 0x7fe7b014c120 con 0x7fe79006aed0 2023-11-24T08:00:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe79006aed0 msgr2=0x7fe79006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe79006aed0 0x7fe79006d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe798009840 tx=0x7fe79801c040 comp rx=0 tx=0).stop 2023-11-24T08:00:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 msgr2=0x7fe7b015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b015ec70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c4d0 tx=0x7fe7a000c7e0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 shutdown_connections 2023-11-24T08:00:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fe79006aed0 0x7fe79006d390 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe7b015ffb0 0x7fe7b014d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b006df20 0x7fe7b015ec70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 --2- 172.21.15.32:0/1764903110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7b0145760 0x7fe7b014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:37.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 >> 172.21.15.32:0/1764903110 conn(0x7fe7b00b1740 msgr2=0x7fe7b00cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:37.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 shutdown_connections 2023-11-24T08:00:37.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:37.909+0000 7fe7b5b26700 1 -- 172.21.15.32:0/1764903110 wait complete. 2023-11-24T08:00:37.913 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-11-24T08:00:37.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-24T08:00:36.558911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220504,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754216,"statfs":{"total":773094113280,"available":772868317184,"internally_reserved":0,"allocated":4431872,"data_stored":2432112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002568"},"pg_stats":[{"pgid":"1.0","version":"35'119","reported_seq":187,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:00:22.141393+0000","last_change":"2023-11-24T08:00:22.141457+0000","last_active":"2023-11-24T08:00:22.141393+0000","last_peered":"2023-11-24T08:00:22.141393+0000","last_clean":"2023-11-24T08:00:22.141393+0000","last_became_active":"2023-11-24T07:58:34.469770+0000","last_became_peered":"2023-11-24T07:58:34.469770+0000","last_unstale":"2023-11-24T08:00:22.141393+0000","last_undegraded":"2023-11-24T08:00:22.141393+0000","last_fullsized":"2023-11-24T08:00:22.141393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'119","last_scrub_stamp":"2023-11-24T08:00:22.141234+0000","last_deep_scrub":"35'119","last_deep_scrub_stamp":"2023-11-24T08:00:22.141234+0000","last_clean_scrub_stamp":"2023-11-24T08:00:22.141234+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:11:45.617528+0000","scrub_duration":0.0069690239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.60899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.80900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36599999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.439},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42499999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.439}]},{"osd":4,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.61599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":746790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.433,"5min":0.433,"15min":0.433},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":10.456,"5min":10.456,"15min":10.456},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":18.024000000000001,"5min":18.024000000000001,"15min":18.024000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":10.531000000000001,"5min":10.531000000000001,"15min":10.531000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":7.3520000000000003,"5min":7.3520000000000003,"15min":7.3520000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":15.095000000000001,"5min":15.095000000000001,"15min":15.095000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":18.227,"5min":18.227,"15min":18.227},"last":0.372}]},{"osd":4,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.494999999999999,"5min":10.494999999999999,"15min":10.494999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":7.3959999999999999,"5min":7.3959999999999999,"15min":7.3959999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":17.966999999999999,"5min":17.966999999999999,"15min":17.966999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.9379999999999997,"5min":7.9379999999999997,"15min":7.9379999999999997},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.9729999999999999,"5min":7.9729999999999999,"15min":7.9729999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":10.564,"5min":10.564,"15min":10.564},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4279999999999999,"5min":7.4279999999999999,"15min":7.4279999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.0009999999999994,"5min":8.0009999999999994,"15min":8.0009999999999994},"last":0.79700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":14.045999999999999,"5min":14.045999999999999,"15min":14.045999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.496}]},{"osd":2,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":7.0759999999999996,"5min":7.0759999999999996,"15min":7.0759999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":13.936999999999999,"5min":13.936999999999999,"15min":13.936999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":13.978999999999999,"5min":13.978999999999999,"15min":13.978999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1059999999999999,"5min":7.1059999999999999,"15min":7.1059999999999999},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:59:36 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":14.013999999999999,"5min":14.013999999999999,"15min":14.013999999999999},"last":0.76200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.626}]},{"osd":2,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.3940000000000001,"5min":7.3940000000000001,"15min":7.3940000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.438}]},{"osd":6,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.372},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39000000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":156422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:37.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T08:00:38.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.048+0000 7f0393dfc700 1 Processor -- start 2023-11-24T08:00:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.048+0000 7f0393dfc700 1 -- start start 2023-11-24T08:00:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0393dfc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f038c15b210 0x7f038c06b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0393dfc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c15b640 con 0x7f038c06e370 2023-11-24T08:00:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0393dfc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c15e9f0 con 0x7f038c0b49a0 2023-11-24T08:00:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0393dfc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c15ec90 con 0x7f038c15b210 2023-11-24T08:00:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0392399700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f038c15b210 0x7f038c06b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0391397700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f038c06e370 0x7f038c06b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32872/0 (socket says 172.21.15.32:32872) 2023-11-24T08:00:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0391397700 1 -- 172.21.15.32:0/3344387914 learned_addr learned my addr 172.21.15.32:0/3344387914 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0392399700 1 --2- 172.21.15.32:0/3344387914 >> v2:172.21.15.32:3300/0 conn(0x7f038c15b210 0x7f038c06b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59394/0 (socket says 172.21.15.32:59394) 2023-11-24T08:00:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0392399700 1 -- 172.21.15.32:0/3344387914 >> v1:172.21.15.123:6789/0 conn(0x7f038c0b49a0 legacy=0x7f038c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0392399700 1 -- 172.21.15.32:0/3344387914 >> v1:172.21.15.32:6789/0 conn(0x7f038c06e370 legacy=0x7f038c06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0392399700 1 -- 172.21.15.32:0/3344387914 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c15ef70 con 0x7f038c15b210 2023-11-24T08:00:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0392399700 1 --2- 172.21.15.32:0/3344387914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c15b210 0x7f038c06b800 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0384009d90 tx=0x7f038400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652449074ddd6103 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.049+0000 7f0390b96700 1 -- 172.21.15.32:0/3344387914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038400e070 con 0x7f038c15b210 2023-11-24T08:00:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0390b96700 1 -- 172.21.15.32:0/3344387914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038400bb90 con 0x7f038c15b210 2023-11-24T08:00:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0390b96700 1 -- 172.21.15.32:0/3344387914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384003a20 con 0x7f038c15b210 2023-11-24T08:00:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 -- 172.21.15.32:0/3344387914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c15b210 msgr2=0x7f038c06b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 --2- 172.21.15.32:0/3344387914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c15b210 0x7f038c06b800 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0384009d90 tx=0x7f038400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 -- 172.21.15.32:0/3344387914 shutdown_connections 2023-11-24T08:00:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 --2- 172.21.15.32:0/3344387914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c15b210 0x7f038c06b800 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 -- 172.21.15.32:0/3344387914 >> 172.21.15.32:0/3344387914 conn(0x7f038c0b28f0 msgr2=0x7f038c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 -- 172.21.15.32:0/3344387914 shutdown_connections 2023-11-24T08:00:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.050+0000 7f0393dfc700 1 -- 172.21.15.32:0/3344387914 wait complete. 2023-11-24T08:00:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.051+0000 7f0393dfc700 1 Processor -- start 2023-11-24T08:00:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.051+0000 7f0393dfc700 1 -- start start 2023-11-24T08:00:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0393dfc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 0x7f038c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0393dfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f038c0b49a0 0x7f038c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0393dfc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f038c15b210 0x7f038c1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0393dfc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c156c60 con 0x7f038c06e370 2023-11-24T08:00:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0393dfc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0cd970 con 0x7f038c15b210 2023-11-24T08:00:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0393dfc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c156610 con 0x7f038c0b49a0 2023-11-24T08:00:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0391b98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 0x7f038c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0391b98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 0x7f038c14d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59396/0 (socket says 172.21.15.32:59396) 2023-11-24T08:00:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0391b98700 1 -- 172.21.15.32:0/4276684764 learned_addr learned my addr 172.21.15.32:0/4276684764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0392399700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f038c15b210 0x7f038c1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.052+0000 7f0391397700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f038c0b49a0 0x7f038c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391b98700 1 -- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f038c0b49a0 msgr2=0x7f038c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391b98700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f038c0b49a0 0x7f038c1f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391b98700 1 -- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f038c15b210 msgr2=0x7f038c1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391b98700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f038c15b210 0x7f038c1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391b98700 1 -- 172.21.15.32:0/4276684764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384009a70 con 0x7f038c06e370 2023-11-24T08:00:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0392399700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f038c15b210 0x7f038c1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391397700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f038c0b49a0 0x7f038c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0391b98700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 0x7f038c14d6e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f038800c4d0 tx=0x7f038800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388011790 con 0x7f038c06e370 2023-11-24T08:00:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03880118f0 con 0x7f038c06e370 2023-11-24T08:00:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c1568f0 con 0x7f038c06e370 2023-11-24T08:00:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388011ca0 con 0x7f038c06e370 2023-11-24T08:00:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.053+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c0cd3f0 con 0x7f038c06e370 2023-11-24T08:00:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.054+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c156f40 con 0x7f038c06e370 2023-11-24T08:00:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.054+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f038801c460 con 0x7f038c06e370 2023-11-24T08:00:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.055+0000 7f03827fc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f037006af00 0x7f037006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.055+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0388089070 con 0x7f038c06e370 2023-11-24T08:00:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.055+0000 7f0391397700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f037006af00 0x7f037006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.056+0000 7f0391397700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f037006af00 0x7f037006d3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f038c0af590 tx=0x7f037c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.060+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03880544b0 con 0x7f038c06e370 2023-11-24T08:00:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.258+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f038c068fb0 con 0x7f038c06e370 2023-11-24T08:00:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.261+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0388057ad0 con 0x7f038c06e370 2023-11-24T08:00:38.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.262+0000 7f03827fc700 1 -- 172.21.15.32:0/4276684764 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f03880039b0 con 0x7f038c06e370 2023-11-24T08:00:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f037006af00 msgr2=0x7f037006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f037006af00 0x7f037006d3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f038c0af590 tx=0x7f037c009040 comp rx=0 tx=0).stop 2023-11-24T08:00:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 msgr2=0x7f038c14d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 0x7f038c14d6e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f038800c4d0 tx=0x7f038800c890 comp rx=0 tx=0).stop 2023-11-24T08:00:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 shutdown_connections 2023-11-24T08:00:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7f037006af00 0x7f037006d3c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f038c15b210 0x7f038c1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f038c06e370 0x7f038c14d6e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 --2- 172.21.15.32:0/4276684764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f038c0b49a0 0x7f038c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 >> 172.21.15.32:0/4276684764 conn(0x7f038c0b28f0 msgr2=0x7f038c0ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:38.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 shutdown_connections 2023-11-24T08:00:38.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.264+0000 7f0393dfc700 1 -- 172.21.15.32:0/4276684764 wait complete. 2023-11-24T08:00:38.278 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:00:38.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.401+0000 7fa15c446700 1 Processor -- start 2023-11-24T08:00:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.401+0000 7fa15c446700 1 -- start start 2023-11-24T08:00:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15c446700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa15415e080 0x7fa1540b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15c446700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa15406c710 0x7fa1540afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15c446700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1540ae430 con 0x7fa1540af4d0 2023-11-24T08:00:38.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15c446700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540af0b0 con 0x7fa15406c710 2023-11-24T08:00:38.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15c446700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540b05a0 con 0x7fa15415e080 2023-11-24T08:00:38.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15a9e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa15406c710 0x7fa1540afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa1599e1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1540af4d0 0x7fa15406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32884/0 (socket says 172.21.15.32:32884) 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa1599e1700 1 -- 172.21.15.32:0/222463246 learned_addr learned my addr 172.21.15.32:0/222463246 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.402+0000 7fa15a1e2700 1 --2- 172.21.15.32:0/222463246 >> v2:172.21.15.123:3300/0 conn(0x7fa15415e080 0x7fa1540b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15a9e3700 1 -- 172.21.15.32:0/222463246 >> v2:172.21.15.123:3300/0 conn(0x7fa15415e080 msgr2=0x7fa1540b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15a9e3700 1 --2- 172.21.15.32:0/222463246 >> v2:172.21.15.123:3300/0 conn(0x7fa15415e080 0x7fa1540b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15a9e3700 1 -- 172.21.15.32:0/222463246 >> v1:172.21.15.32:6789/0 conn(0x7fa1540af4d0 legacy=0x7fa15406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15a9e3700 1 -- 172.21.15.32:0/222463246 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540ae710 con 0x7fa15406c710 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa1591e0700 1 -- 172.21.15.32:0/222463246 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034073224 0 0) 0x7fa1540ae430 con 0x7fa1540af4d0 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15a1e2700 1 --2- 172.21.15.32:0/222463246 >> v2:172.21.15.123:3300/0 conn(0x7fa15415e080 0x7fa1540b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15a9e3700 1 --2- 172.21.15.32:0/222463246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15406c710 0x7fa1540afb90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa154072a90 tx=0x7fa14c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0ab7c1cb41e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa1591e0700 1 -- 172.21.15.32:0/222463246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00e070 con 0x7fa15406c710 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa1591e0700 1 -- 172.21.15.32:0/222463246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa14c00bb90 con 0x7fa15406c710 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa1591e0700 1 -- 172.21.15.32:0/222463246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c012430 con 0x7fa15406c710 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15c446700 1 -- 172.21.15.32:0/222463246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15406c710 msgr2=0x7fa1540afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.403+0000 7fa15c446700 1 --2- 172.21.15.32:0/222463246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15406c710 0x7fa1540afb90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa154072a90 tx=0x7fa14c00b330 comp rx=0 tx=0).stop 2023-11-24T08:00:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.404+0000 7fa15c446700 1 -- 172.21.15.32:0/222463246 shutdown_connections 2023-11-24T08:00:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.404+0000 7fa15c446700 1 --2- 172.21.15.32:0/222463246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa15406c710 0x7fa1540afb90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.404+0000 7fa15c446700 1 --2- 172.21.15.32:0/222463246 >> v2:172.21.15.123:3300/0 conn(0x7fa15415e080 0x7fa1540b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.404+0000 7fa15c446700 1 -- 172.21.15.32:0/222463246 >> 172.21.15.32:0/222463246 conn(0x7fa154071b90 msgr2=0x7fa154071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:38.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.404+0000 7fa15c446700 1 -- 172.21.15.32:0/222463246 shutdown_connections 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.404+0000 7fa15c446700 1 -- 172.21.15.32:0/222463246 wait complete. 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.405+0000 7fa15c446700 1 Processor -- start 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.405+0000 7fa15c446700 1 -- start start 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa15406c710 0x7fa15415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 0x7fa1541488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415e080 0x7fa1541561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa154154e40 con 0x7fa1540af4d0 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa154155120 con 0x7fa15406c710 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154155400 con 0x7fa15415e080 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 0x7fa1541488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 0x7fa1541488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59412/0 (socket says 172.21.15.32:59412) 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 -- 172.21.15.32:0/1868171273 learned_addr learned my addr 172.21.15.32:0/1868171273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15a1e2700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa15406c710 0x7fa15415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15a9e3700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415e080 0x7fa1541561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 -- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415e080 msgr2=0x7fa1541561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415e080 0x7fa1541561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 -- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa15406c710 msgr2=0x7fa15415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa15406c710 0x7fa15415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 -- 172.21.15.32:0/1868171273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c009a70 con 0x7fa1540af4d0 2023-11-24T08:00:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa1599e1700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 0x7fa1541488b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa15000c540 tx=0x7fa15000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150004c80 con 0x7fa1540af4d0 2023-11-24T08:00:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa150004de0 con 0x7fa1540af4d0 2023-11-24T08:00:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150007330 con 0x7fa1540af4d0 2023-11-24T08:00:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1541556e0 con 0x7fa1540af4d0 2023-11-24T08:00:38.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.406+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa154155ca0 con 0x7fa1540af4d0 2023-11-24T08:00:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.407+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1540753f0 con 0x7fa1540af4d0 2023-11-24T08:00:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.408+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa150007490 con 0x7fa1540af4d0 2023-11-24T08:00:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.408+0000 7fa146ffd700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13806aed0 0x7fa13806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.408+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa150088f30 con 0x7fa1540af4d0 2023-11-24T08:00:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.409+0000 7fa15a1e2700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13806aed0 0x7fa13806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:38.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.409+0000 7fa15a1e2700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13806aed0 0x7fa13806d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c940 tx=0x7fa148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:38.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.413+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1500542e0 con 0x7fa1540af4d0 2023-11-24T08:00:38.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.660+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa154067f60 con 0x7fa1540af4d0 2023-11-24T08:00:38.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.660+0000 7fa146ffd700 1 -- 172.21.15.32:0/1868171273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fa150057900 con 0x7fa1540af4d0 2023-11-24T08:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-24T08:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/384450500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":384450500},{"type":"v1","addr":"172.21.15.123:6836","nonce":384450500}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6834/1778777087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1778777087},{"type":"v1","addr":"172.21.15.32:6834","nonce":1778777087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1261325057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1261325057},{"type":"v1","addr":"172.21.15.123:6837","nonce":1261325057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1091685387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6833","nonce":1091685387},{"type":"v1","addr":"172.21.15.32:6835","nonce":1091685387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-11-24T08:00:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13806aed0 msgr2=0x7fa13806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13806aed0 0x7fa13806d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c940 tx=0x7fa148008040 comp rx=0 tx=0).stop 2023-11-24T08:00:38.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 msgr2=0x7fa1541488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 0x7fa1541488b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa15000c540 tx=0x7fa15000c850 comp rx=0 tx=0).stop 2023-11-24T08:00:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 shutdown_connections 2023-11-24T08:00:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:6832/1302929475,v1:172.21.15.123:6833/1302929475] conn(0x7fa13806aed0 0x7fa13806d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa15415e080 0x7fa1541561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa15406c710 0x7fa15415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 --2- 172.21.15.32:0/1868171273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1540af4d0 0x7fa1541488b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 >> 172.21.15.32:0/1868171273 conn(0x7fa154071b90 msgr2=0x7fa1540b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 shutdown_connections 2023-11-24T08:00:38.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-11-24T08:00:38.662+0000 7fa15c446700 1 -- 172.21.15.32:0/1868171273 wait complete. 2023-11-24T08:00:38.667 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2023-11-24T08:00:38.676 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T08:00:38.676 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T08:00:38.677 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:00:38.677 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.686 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:00:38.686 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:00:38.686 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.734 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:00:38.735 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:00:38.735 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.746 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:00:38.746 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:00:38.747 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.797 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:00:38.798 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T08:00:38.798 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T08:00:38.798 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.832 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T08:00:38.833 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T08:00:38.833 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.902 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T08:00:38.903 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T08:00:38.903 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:38.981 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T08:00:38.981 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T08:00:38.981 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.060 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T08:00:39.061 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T08:00:39.061 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.105 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T08:00:39.106 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T08:00:39.106 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.142 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T08:00:39.142 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T08:00:39.142 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.219 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T08:00:39.220 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T08:00:39.220 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.296 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T08:00:39.296 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T08:00:39.297 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T08:00:39.297 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.320 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T08:00:39.321 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T08:00:39.321 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.366 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T08:00:39.366 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T08:00:39.367 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T08:00:39.367 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.378 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T08:00:39.378 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T08:00:39.378 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.418 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T08:00:39.419 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T08:00:39.419 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:39.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-24T08:00:39.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-11-24T08:00:39.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-24T08:00:39.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-11-24T08:00:39.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-11-24T08:00:39.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-24T08:00:39.472 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T08:00:39.472 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T08:00:39.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T08:00:39.509 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T08:00:39.509 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T08:00:39.772 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T08:00:39.772 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T08:00:39.859 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T08:00:39.859 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T08:00:39.943 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T08:00:39.943 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T08:00:40.019 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T08:00:40.020 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T08:00:40.184 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T08:00:40.184 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T08:00:40.283 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T08:00:40.284 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T08:00:40.360 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T08:00:40.361 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T08:00:40.442 INFO:tasks.ceph:Archiving mon data... 2023-11-24T08:00:40.444 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/data/mon.a.tgz 2023-11-24T08:00:40.444 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-11-24T08:00:40.463 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.Ighm1OvJ7j 2023-11-24T08:00:40.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Ighm1OvJ7j 2023-11-24T08:00:40.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.Ighm1OvJ7j 2023-11-24T08:00:40.681 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.Ighm1OvJ7j is 285KB 2023-11-24T08:00:40.780 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.Ighm1OvJ7j 2023-11-24T08:00:40.798 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/data/mon.b.tgz 2023-11-24T08:00:40.798 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-11-24T08:00:40.818 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.KFtAsvduze 2023-11-24T08:00:40.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KFtAsvduze 2023-11-24T08:00:40.971 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.KFtAsvduze 2023-11-24T08:00:41.057 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.KFtAsvduze is 384KB 2023-11-24T08:00:41.154 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.KFtAsvduze 2023-11-24T08:00:41.171 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/data/mon.c.tgz 2023-11-24T08:00:41.171 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-11-24T08:00:41.227 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.L9kNjCRVMH 2023-11-24T08:00:41.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.L9kNjCRVMH 2023-11-24T08:00:41.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.L9kNjCRVMH 2023-11-24T08:00:41.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:41.418 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:41.523 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.L9kNjCRVMH is 384KB 2023-11-24T08:00:41.620 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.L9kNjCRVMH 2023-11-24T08:00:41.635 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T08:00:41.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:00:41.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:00:41.701 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T08:00:41.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:00:41.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:00:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:00:41.778 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:00:41.783 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T08:00:41.785 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/remote/smithi032/crash 2023-11-24T08:00:41.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:00:41.817 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/remote/smithi123/crash 2023-11-24T08:00:41.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:00:41.849 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T08:00:41.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:41.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:41.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:41.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:42.000 INFO:tasks.ceph:Compressing logs... 2023-11-24T08:00:42.001 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:00:42.017 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:00:42.086 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110512.log 2023-11-24T08:00:42.087 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111301.log 2023-11-24T08:00:42.087 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.3.110512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111305.log 2023-11-24T08:00:42.087 INFO:teuthology.orchestra.run.smithi123.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110512.log.gz 2023-11-24T08:00:42.088 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.1.111301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111301.log.gz 2023-11-24T08:00:42.088 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103512.log 2023-11-24T08:00:42.088 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T08:00:42.088 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.1.111305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111305.log.gz 2023-11-24T08:00:42.089 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103512.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:00:42.090 INFO:teuthology.orchestra.run.smithi123.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103512.log.gz 2023-11-24T08:00:42.090 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.109905.log 2023-11-24T08:00:42.090 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103557.log 2023-11-24T08:00:42.090 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.1.109905.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.1.109905.log.gz--verbose 2023-11-24T08:00:42.091 INFO:teuthology.orchestra.run.smithi123.stderr:-- /var/log/ceph/ceph-osd.6.log 2023-11-24T08:00:42.091 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103647.log 2023-11-24T08:00:42.091 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103557.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103420.log 2023-11-24T08:00:42.091 INFO:teuthology.orchestra.run.smithi123.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103557.log.gz 2023-11-24T08:00:42.091 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:00:42.091 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103647.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103465.log 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103420.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103647.log.gz 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103420.log.gz 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103465.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103465.log.gz 2023-11-24T08:00:42.092 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103602.log 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103602.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103602.log.gz 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mgr.x.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103700.log 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.103700.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103700.log.gz 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110516.log 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.3.110516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110516.log.gz 2023-11-24T08:00:42.093 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T08:00:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T08:00:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T08:00:42.103 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.b.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T08:00:42.103 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.109897.log 2023-11-24T08:00:42.114 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.1.109897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.109897.log.gz 2023-11-24T08:00:42.115 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103373.log 2023-11-24T08:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118959.log 2023-11-24T08:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118591.log 2023-11-24T08:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111285.log 2023-11-24T08:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118959.log.gz 2023-11-24T08:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118821.log 2023-11-24T08:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118591.log.gz 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112565.log 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111285.log: /var/log/ceph/ceph-client.admin.118821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109332.log 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111285.log.gz 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.112565.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118821.log.gz 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116548.log 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109332.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112565.log.gz 2023-11-24T08:00:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.117672.log 2023-11-24T08:00:42.122 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109332.log.gz 2023-11-24T08:00:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120734.log 2023-11-24T08:00:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.117672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109516.log 2023-11-24T08:00:42.122 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116548.log.gz 2023-11-24T08:00:42.123 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.117672.log.gz 2023-11-24T08:00:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118480.log 2023-11-24T08:00:42.123 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120734.log.gz 2023-11-24T08:00:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112343.log 2023-11-24T08:00:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109516.log: gzip -5 /var/log/ceph/ceph-client.admin.118480.log: --verbose -- /var/log/ceph/ceph-client.admin.118024.log 2023-11-24T08:00:42.123 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109516.log.gz 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.112343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117160.log 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112343.log.gz 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.118480.log.gz 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103485.log 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117160.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118024.log.gz 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117252.log 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117160.log.gz 2023-11-24T08:00:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.103485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.117668.log 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103485.log.gz 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110063.log 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.117668.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117252.log.gz 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109767.log 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.117668.log.gz 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115590.log 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 88.6%--verbose -- /var/log/ceph/ceph-osd.2.log -- replaced with /var/log/ceph/ceph-client.admin.110063.log.gz 2023-11-24T08:00:42.125 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109767.log: /var/log/ceph/ceph-client.admin.115590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113360.log 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109767.log.gz 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.115590.log.gz 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120503.log 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120273.log 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120503.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.113360.log.gz 2023-11-24T08:00:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111515.log 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120503.log.gz 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114013.log 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120273.log: /var/log/ceph/ceph-client.admin.111515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117824.log 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120273.log.gz 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114013.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111515.log.gz 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96885.log 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114013.log.gz 2023-11-24T08:00:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116974.log 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117824.log.gz 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.96885.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96885.log.gz 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120319.log 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116974.log.gz 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119905.log 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.112996.log 2023-11-24T08:00:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119905.log: 88.8% 85.3% -- replaced with /var/log/ceph/ceph-client.admin.120319.log.gz -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109675.log 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119905.log.gz 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.112996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.112996.log.gz 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120135.log 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116732.log 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113498.log 2023-11-24T08:00:42.129 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109675.log.gz 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120135.log.gz 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109228.log 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115452.log 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113498.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116732.log.gz 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111469.log 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113498.log.gz 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115452.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109228.log.gz 2023-11-24T08:00:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114105.log 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111469.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115452.log.gz 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115636.log 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114105.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111469.log.gz 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118296.log 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114105.log.gz 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114893.log 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119792.log 2023-11-24T08:00:42.131 INFO:teuthology.orchestra.run.smithi032.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.115636.log.gz 2023-11-24T08:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114893.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118296.log.gz 2023-11-24T08:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117414.log 2023-11-24T08:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119792.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114893.log.gz 2023-11-24T08:00:42.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120457.log 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119792.log.gz 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116928.log 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117414.log: /var/log/ceph/ceph-client.admin.120457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120365.log 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117414.log.gz 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120457.log.gz 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110803.log 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114709.log 2023-11-24T08:00:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120365.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116928.log.gz 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110751.log 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120365.log.gz 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110803.log.gz 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114709.log: /var/log/ceph/ceph-osd.0.log 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114437.log 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114709.log.gz 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.0.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110751.log.gz 2023-11-24T08:00:42.134 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110057.log 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120089.log 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109629.log 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114437.log.gz 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120089.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110057.log.gz 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120824.log 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109629.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120089.log.gz 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119208.log 2023-11-24T08:00:42.135 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117022.log 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109629.log.gz 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120824.log.gz 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119208.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114529.log 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117022.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119208.log.gz 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.a.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117022.log.gzgzip 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.118729.log 2023-11-24T08:00:42.136 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115088.log 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118729.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114529.log.gz 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115088.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118729.log.gz 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.115088.log.gz 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115866.log 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116686.log 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111699.log 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115866.log.gz 2023-11-24T08:00:42.137 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117870.log 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109284.log 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116686.log.gz 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115544.log 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109284.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117870.log.gz 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.111699.log.gz 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115544.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109284.log.gz 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115544.log.gz 2023-11-24T08:00:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114243.log 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111377.log 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114243.log.gz 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119069.log 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111377.log: /var/log/ceph/ceph-client.admin.119069.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111377.log.gz 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119069.log.gz 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117114.log 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116270.log 2023-11-24T08:00:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117114.log: /var/log/ceph/ceph-client.admin.116270.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117114.log.gz 2023-11-24T08:00:42.140 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116270.log.gz 2023-11-24T08:00:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116456.log 2023-11-24T08:00:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116096.log 2023-11-24T08:00:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118208.log 2023-11-24T08:00:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116096.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116456.log.gz 2023-11-24T08:00:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118208.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116096.log.gz 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118208.log.gz 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113683.log 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110061.log 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110752.log 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110061.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113683.log.gz 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110752.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110061.log.gz 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110752.log.gz 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108880.log 2023-11-24T08:00:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116410.log 2023-11-24T08:00:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.108880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110641.log 2023-11-24T08:00:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116410.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108880.log.gz 2023-11-24T08:00:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110641.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116410.log.gz 2023-11-24T08:00:42.142 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110641.log.gz 2023-11-24T08:00:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117068.log 2023-11-24T08:00:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119162.log 2023-11-24T08:00:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117068.log: /var/log/ceph/ceph-client.admin.119162.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117068.log.gz 2023-11-24T08:00:42.143 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119162.log.gz 2023-11-24T08:00:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111423.log 2023-11-24T08:00:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113775.log 2023-11-24T08:00:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111423.log: /var/log/ceph/ceph-client.admin.113775.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111423.log.gz 2023-11-24T08:00:42.143 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113775.log.gz 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110046.log 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110046.log: /var/log/ceph/ceph-mds.c.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110046.log.gz 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117778.log 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113452.log 2023-11-24T08:00:42.144 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114197.log 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113452.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117778.log.gz 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114197.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113452.log.gz 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114197.log.gz 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118683.log 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120687.log 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118683.log: /var/log/ceph/ceph-client.admin.120687.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118683.log.gz 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120687.log.gz 2023-11-24T08:00:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110060.log 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114151.log 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117298.log 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110060.log.gz/var/log/ceph/ceph-client.admin.114151.log: 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117298.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114151.log.gz 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117298.log.gz 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110757.log 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115728.log 2023-11-24T08:00:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118545.log 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115728.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110757.log.gz 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118545.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115728.log.gz 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118545.log.gz 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116828.log 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117344.log 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116362.log 2023-11-24T08:00:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119005.log 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116828.log.gz 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116362.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.117344.log.gz 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116362.log.gz/var/log/ceph/ceph-client.admin.119005.log: 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115360.log 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119005.log.gz 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114575.log 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110424.log 2023-11-24T08:00:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114575.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115360.log.gz 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115774.log 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110424.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114575.log.gz 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110433.log 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115774.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110424.log.gz 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113000.log 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110433.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115774.log.gz 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110433.log.gz 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.113000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113000.log.gz 2023-11-24T08:00:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120641.log 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111101.log 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119304.log 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120641.log.gz 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111101.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111101.log.gz 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118162.log 2023-11-24T08:00:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115406.log 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119304.log.gz 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118162.log: /var/log/ceph/ceph-client.admin.115406.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118162.log.gz 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115406.log.gz 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118342.log 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120181.log 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110065.log 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.118342.log.gz 2023-11-24T08:00:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120227.log 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119116.log 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110065.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120181.log.gz 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120227.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110065.log.gz 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119116.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120227.log.gz 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119116.log.gz 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111193.log 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111009.log 2023-11-24T08:00:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119997.log 2023-11-24T08:00:42.153 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111193.log.gz 2023-11-24T08:00:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111147.log 2023-11-24T08:00:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115314.log 2023-11-24T08:00:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119997.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111009.log.gz 2023-11-24T08:00:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111147.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119997.log.gz 2023-11-24T08:00:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115314.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111147.log.gz 2023-11-24T08:00:42.154 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115314.log.gz 2023-11-24T08:00:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114621.log 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113268.log 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109011.log 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114621.log.gz 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108771.log 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113729.log 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109011.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113268.log.gz 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.108771.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109011.log.gz 2023-11-24T08:00:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113729.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108771.log.gz 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113729.log.gz 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115180.log 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118116.log 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114801.log 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115180.log.gz 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111331.log 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120411.log 2023-11-24T08:00:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114801.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118116.log.gz 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111331.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114801.log.gz 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120411.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111331.log.gz 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120411.log.gz 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118867.log 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112335.log 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119838.log 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118867.log.gz 2023-11-24T08:00:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.112335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120043.log 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112335.log.gz 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113314.log 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119838.log: /var/log/ceph/ceph-client.admin.120043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109470.log 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119838.log.gz 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113314.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120043.log.gz 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109470.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113314.log.gz 2023-11-24T08:00:42.158 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109470.log.gz 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109583.log 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118388.log 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109583.log.gz 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.118388.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118388.log.gz 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109058.log 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109721.log 2023-11-24T08:00:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109058.log: /var/log/ceph/ceph-client.admin.109721.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109058.log.gz 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109721.log.gz 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115682.log 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115958.log 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115268.log 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115958.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115682.log.gz 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115958.log.gz 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115268.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115268.log.gz 2023-11-24T08:00:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119404.log 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113637.log 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114755.log 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113637.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119404.log.gz 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114755.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113637.log.gz 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114755.log.gz 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113821.log 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110064.log 2023-11-24T08:00:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119350.log 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110064.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113821.log.gz 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119350.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110064.log.gz 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119350.log.gz 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110432.log 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103177.log 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115134.log 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.103177.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110432.log.gz 2023-11-24T08:00:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115134.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.103177.log.gz 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115134.log.gz 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111853.log 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116640.log 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116502.log 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116640.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111853.log.gz 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116502.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116640.log.gz 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118637.log 2023-11-24T08:00:42.163 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116502.log.gz 2023-11-24T08:00:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113406.log 2023-11-24T08:00:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118637.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118637.log.gz 2023-11-24T08:00:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113406.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113406.log.gz 2023-11-24T08:00:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118434.log 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110798.log 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118070.log 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110798.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118434.log.gz 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118070.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110798.log.gz 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118070.log.gz 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119746.log 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113544.log 2023-11-24T08:00:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113967.log 2023-11-24T08:00:42.166 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119746.log.gz 2023-11-24T08:00:42.166 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113544.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113544.log.gz 2023-11-24T08:00:42.167 INFO:teuthology.orchestra.run.smithi123.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T08:00:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.103373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103373.log.gz 2023-11-24T08:00:42.167 INFO:teuthology.orchestra.run.smithi123.stderr: 95.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T08:00:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109378.log 2023-11-24T08:00:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120778.log 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113967.log.gz 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109378.log.gz 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113590.log 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109104.log 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120778.log.gz 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113590.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113590.log.gz/var/log/ceph/ceph-client.admin.109104.log: 2023-11-24T08:00:42.180 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109104.log.gz 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110753.log 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119951.log 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118913.log 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1%/var/log/ceph/ceph-client.admin.119951.log: -- replaced with /var/log/ceph/ceph-client.admin.110753.log.gz 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118913.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119951.log.gz 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118913.log.gz 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110434.log 2023-11-24T08:00:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116050.log 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110426.log 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110434.log.gz 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116050.log.gz 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111561.log 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108965.log 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111561.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110426.log.gz 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111561.log.gz 2023-11-24T08:00:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.108965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114996.log 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108965.log.gz 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114996.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114996.log.gz 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117206.log 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116316.log 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109813.log 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116316.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117206.log.gz 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109813.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116316.log.gz 2023-11-24T08:00:42.183 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109813.log.gz 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115820.log 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116004.log 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114847.log 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115820.log: /var/log/ceph/ceph-client.admin.116004.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115820.log.gz 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114847.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116004.log.gz 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114847.log.gz 2023-11-24T08:00:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116780.log 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111653.log 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116882.log 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111653.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116780.log.gz 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116882.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111653.log.gz 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116882.log.gz 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115912.log 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109173.log 2023-11-24T08:00:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115912.log: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111239.log 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109173.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115912.log.gz 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117916.log 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109173.log.gz 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119257.log 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111239.log.gz 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.117916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115498.log 2023-11-24T08:00:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.119257.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117916.log.gz 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110754.log 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119257.log.gz 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111607.log 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110754.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115498.log.gz 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120549.log 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111607.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110754.log.gz 2023-11-24T08:00:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111055.log 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.111607.log.gz 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109859.log 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120549.log.gz 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.111055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116186.log 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115042.log 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109859.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111055.log.gz 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116186.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T08:00:42.188 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109859.log.gz 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.115042.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116186.log.gz 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115042.log.gz 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114059.log 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114483.log 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114289.log 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114483.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114059.log.gz 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.114289.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114483.log.gz 2023-11-24T08:00:42.189 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114289.log.gz 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110427.log 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120595.log 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116594.log 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.120595.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110427.log.gz 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120595.log.gz 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113921.log 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.116594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109424.log 2023-11-24T08:00:42.190 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0%/var/log/ceph/ceph-client.admin.113921.log: -- replaced with /var/log/ceph/ceph-client.admin.116594.log.gz 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113921.log.gz 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103615.log 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.109424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110430.log 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2%/var/log/ceph/ceph-client.admin.103615.log: -- replaced with /var/log/ceph/ceph-client.admin.109424.log.gz 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103615.log.gz 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110430.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110430.log.gz 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113875.log 2023-11-24T08:00:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113222.log 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113875.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113875.log.gz 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.113222.log: /var/log/ceph/ceph-mgr.y.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113222.log.gz 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110423.log 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118775.log 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110423.log: /var/log/ceph/ceph-client.admin.118775.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110423.log.gz 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118775.log.gz 2023-11-24T08:00:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110066.log 2023-11-24T08:00:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.110066.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110066.log.gz 2023-11-24T08:00:42.193 INFO:teuthology.orchestra.run.smithi032.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T08:00:42.193 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T08:00:42.193 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T08:00:42.194 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T08:00:42.194 INFO:teuthology.orchestra.run.smithi123.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T08:00:42.194 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T08:00:42.194 INFO:teuthology.orchestra.run.smithi123.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T08:00:42.194 INFO:teuthology.orchestra.run.smithi123.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T08:00:42.204 INFO:teuthology.orchestra.run.smithi032.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T08:00:42.230 INFO:teuthology.orchestra.run.smithi123.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T08:00:42.231 INFO:teuthology.orchestra.run.smithi123.stderr: 2023-11-24T08:00:42.231 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m0.175s 2023-11-24T08:00:42.231 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m0.805s 2023-11-24T08:00:42.232 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.044s 2023-11-24T08:00:42.320 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T08:00:42.321 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-11-24T08:00:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m0.290s 2023-11-24T08:00:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m0.709s 2023-11-24T08:00:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.156s 2023-11-24T08:00:42.322 INFO:tasks.ceph:Archiving logs... 2023-11-24T08:00:42.322 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/remote/smithi032/log 2023-11-24T08:00:42.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:00:42.660 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/remote/smithi123/log 2023-11-24T08:00:42.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:00:42.788 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T08:00:42.825 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T08:00:42.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:00:42.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:00:42.909 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T08:00:42.922 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T08:00:42.923 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:00:42.926 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:00:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-11-24T08:00:42.941 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-11-24T08:00:42.945 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:00:42.945 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-11-24T08:00:42.945 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 +2733us[+2722us] +/- 109ms 2023-11-24T08:00:42.945 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -711us[ -722us] +/- 76ms 2023-11-24T08:00:42.945 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -1162us[-1173us] +/- 76ms 2023-11-24T08:00:42.945 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:42.946 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:00:42.947 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-11-24T08:00:42.947 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +2733us[+2733us] +/- 109ms 2023-11-24T08:00:42.947 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -701us[ -692us] +/- 76ms 2023-11-24T08:00:42.947 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -1147us[-1139us] +/- 76ms 2023-11-24T08:00:42.947 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:00:42.948 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T08:00:42.957 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T08:00:42.958 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T08:00:42.967 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:00:43.088 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:00:43.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:00:43.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:43.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:00:43.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:43.245 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T08:00:43.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T08:00:43.265 INFO:teuthology.task.internal:Duration was 880.036776 seconds 2023-11-24T08:00:43.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T08:00:43.276 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T08:00:43.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:00:43.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:00:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:00:43.331 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:00:43.536 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T08:00:43.536 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-11-24T08:00:43.536 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:00:43.560 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2023-11-24T08:00:43.560 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:00:43.581 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T08:00:43.581 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:00:43.603 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:00:43.662 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T08:00:43.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:00:43.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:00:43.777 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T08:00:43.789 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T08:00:43.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:00:43.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:00:43.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T08:00:43.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:00:43.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:00:43.884 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-11-24T08:00:43.910 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2023-11-24T08:00:43.924 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:00:43.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:43.956 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:00:43.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:43.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T08:00:43.991 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T08:00:43.991 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/remote/smithi032 2023-11-24T08:00:43.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:00:44.035 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578/remote/smithi123 2023-11-24T08:00:44.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:00:44.073 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T08:00:44.074 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:00:44.076 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:00:44.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T08:00:44.136 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T08:00:44.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T08:00:44.147 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T08:00:44.148 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:00:44.150 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:00:44.165 INFO:teuthology.orchestra.run.smithi032.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:00 /home/ubuntu/cephtest 2023-11-24T08:00:44.181 INFO:teuthology.orchestra.run.smithi123.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:00 /home/ubuntu/cephtest 2023-11-24T08:00:44.182 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T08:00:44.196 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T08:00:44.219 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T08:00:44.246 DEBUG:teuthology.nuke:shortname: smithi032 2023-11-24T08:00:44.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:00:44.272 DEBUG:teuthology.nuke:shortname: smithi123 2023-11-24T08:00:44.272 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:00:44.276 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:40:55.545724', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T08:00:44.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:40:55.544288', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T08:00:44.329 INFO:teuthology.orchestra.console.smithi032:Power off 2023-11-24T08:00:44.330 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:00:44.356 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-11-24T08:00:44.357 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:44.376 INFO:teuthology.orchestra.console.smithi123:Power off 2023-11-24T08:00:44.377 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:00:44.388 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-11-24T08:00:44.402 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-11-24T08:00:44.402 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:44.426 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-11-24T08:00:44.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-24T08:00:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-11-24T08:00:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-24T08:00:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-11-24T08:00:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-11-24T08:00:44.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-24T08:00:48.389 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:48.429 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:48.500 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-11-24T08:00:48.540 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-11-24T08:00:52.502 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:52.541 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:52.614 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-11-24T08:00:52.652 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-11-24T08:00:52.716 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-11-24T08:00:56.654 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:00:56.764 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-11-24T08:00:56.865 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-11-24T08:00:56.970 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} duration: 880.0367760658264 failure_reason: 'Test failure: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=906c694f5a854172af02d98e4268a9f9 status: fail success: false 2023-11-24T08:00:56.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T08:00:57.044 INFO:teuthology.run:FAIL